Jul 11 07:36:25.873998 ip-10-0-132-10 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 11 07:36:25.874007 ip-10-0-132-10 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 11 07:36:25.874014 ip-10-0-132-10 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 11 07:36:25.874253 ip-10-0-132-10 systemd[1]: Failed to start Kubernetes Kubelet. Jul 11 07:36:36.118580 ip-10-0-132-10 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 11 07:36:36.118598 ip-10-0-132-10 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 7aee839c739144a69d7db4274782274a -- Jul 11 07:38:30.033972 ip-10-0-132-10 systemd[1]: Starting Kubernetes Kubelet... Jul 11 07:38:30.434756 ip-10-0-132-10 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 07:38:30.434756 ip-10-0-132-10 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 11 07:38:30.434756 ip-10-0-132-10 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 07:38:30.434756 ip-10-0-132-10 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 11 07:38:30.434756 ip-10-0-132-10 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 07:38:30.435777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.435674 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 11 07:38:30.441377 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441357 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.441377 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441374 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.441377 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441379 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.441377 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441383 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441387 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441391 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441395 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441399 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441403 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441407 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441412 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441416 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441420 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441423 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441434 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441438 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441442 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441446 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441449 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441454 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441459 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441462 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.441613 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441466 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441470 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441474 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441478 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441481 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441485 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441489 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441493 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441497 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441501 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441505 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441509 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441513 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441517 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441521 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441526 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441531 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441535 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441540 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441544 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.442367 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441548 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441552 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441556 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441560 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441564 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441570 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441576 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441580 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441585 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441589 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441594 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441599 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441603 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441607 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441612 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441616 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441620 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441627 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441632 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.443241 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441636 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441641 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441646 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441650 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441655 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441659 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441664 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441672 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441678 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441683 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441689 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441694 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441698 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441716 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441720 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441724 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441728 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441732 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441737 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441741 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.443960 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441745 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441748 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441752 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441757 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441762 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441766 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441770 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441774 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441778 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441782 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441787 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441791 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441795 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441799 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441803 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441807 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441811 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.441815 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.442967 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.444453 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.442978 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.442983 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.442989 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.442994 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443000 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443005 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443010 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443014 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443018 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443022 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443026 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443029 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443033 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443040 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443045 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443050 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443056 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443060 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443064 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443068 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.445231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443072 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443075 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443079 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443083 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443087 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443093 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443097 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443101 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443105 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443109 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443113 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443117 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443120 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443125 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443129 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443133 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443137 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443142 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443147 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443150 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.446079 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443154 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443158 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443162 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443166 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443172 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443176 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443180 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443184 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443188 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443192 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443196 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443200 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443203 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443207 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443211 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443215 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443219 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443224 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443228 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443231 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.446840 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443235 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443239 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443242 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443246 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443250 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443254 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443258 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443262 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443266 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443270 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443275 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443280 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443285 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443289 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443293 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443296 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443300 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443306 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443310 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.447359 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443316 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443322 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443327 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443331 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443335 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443339 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443345 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443351 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443356 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443360 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443365 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443369 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443373 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443377 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443380 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443384 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443388 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443392 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.443396 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.447908 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444592 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444603 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444608 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444613 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444618 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444622 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444627 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444631 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444635 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444639 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444643 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444650 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444655 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444661 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444665 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444669 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444673 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444677 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444681 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.448521 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444686 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444690 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444694 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444698 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444719 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444723 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444728 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444736 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444742 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444746 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444751 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444755 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444760 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444764 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444769 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444773 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444777 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444782 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444787 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444792 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.448989 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444796 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444800 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444804 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444808 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444813 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444816 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444821 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444824 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444829 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444833 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444837 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444841 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444845 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444849 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444853 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444857 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444861 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444865 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444869 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444873 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.449502 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444877 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444881 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444885 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444889 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444893 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444898 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444902 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444906 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444910 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444914 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444918 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444923 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444927 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444931 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444935 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444940 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444945 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444949 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444953 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444957 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.450006 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444962 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444965 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444969 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444973 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444977 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444983 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444989 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444994 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.444998 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445002 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445007 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445010 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445014 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445018 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445022 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445026 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445031 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445035 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445039 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.445043 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.450473 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445136 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445146 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445155 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445161 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445176 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445181 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445187 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445193 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445198 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445203 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445208 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445214 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445219 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445224 2572 flags.go:64] FLAG: --cgroup-root="" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445228 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445232 2572 flags.go:64] FLAG: --client-ca-file="" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445237 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445241 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445249 2572 flags.go:64] FLAG: --cluster-domain="" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445253 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445258 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445263 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445269 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445275 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 11 07:38:30.450972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445280 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445285 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445290 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445295 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445300 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445305 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445310 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445314 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445321 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445326 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445330 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445335 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445340 2572 flags.go:64] FLAG: --enable-server="true" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445345 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445352 2572 flags.go:64] FLAG: --event-burst="100" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445357 2572 flags.go:64] FLAG: --event-qps="50" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445361 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445366 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445370 2572 flags.go:64] FLAG: --eviction-hard="" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445376 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445381 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445386 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445391 2572 flags.go:64] FLAG: --eviction-soft="" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445399 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445404 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 11 07:38:30.451533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445408 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445413 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445417 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445422 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445427 2572 flags.go:64] FLAG: --feature-gates="" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445433 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445438 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445442 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445447 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445452 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445457 2572 flags.go:64] FLAG: --help="false" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445461 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445466 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445471 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445476 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445481 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445486 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445491 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445495 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445500 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445505 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445510 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445515 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445520 2572 flags.go:64] FLAG: --kube-reserved="" Jul 11 07:38:30.452171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445524 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445528 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445533 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445541 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445545 2572 flags.go:64] FLAG: --lock-file="" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445550 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445556 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445561 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445570 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445574 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445578 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445583 2572 flags.go:64] FLAG: --logging-format="text" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445588 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445593 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445598 2572 flags.go:64] FLAG: --manifest-url="" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445602 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445608 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445613 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445620 2572 flags.go:64] FLAG: --max-pods="110" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445625 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445630 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445634 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445639 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445644 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445648 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 11 07:38:30.452807 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445652 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445663 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445667 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445672 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445677 2572 flags.go:64] FLAG: --pod-cidr="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445682 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445691 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445696 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445716 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445721 2572 flags.go:64] FLAG: --port="10250" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445728 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445733 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ea9f3f8096663db5" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445738 2572 flags.go:64] FLAG: --qos-reserved="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445743 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445751 2572 flags.go:64] FLAG: --register-node="true" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445755 2572 flags.go:64] FLAG: --register-with-taints="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445761 2572 flags.go:64] FLAG: --registry-burst="10" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445765 2572 flags.go:64] FLAG: --registry-qps="5" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445770 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445774 2572 flags.go:64] FLAG: --reserved-memory="" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445780 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445785 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445790 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445795 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445799 2572 flags.go:64] FLAG: --runonce="false" Jul 11 07:38:30.453394 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445803 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445808 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445813 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445818 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445822 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445827 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445832 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445837 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445841 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445846 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445850 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445855 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445860 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445866 2572 flags.go:64] FLAG: --system-cgroups="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445870 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445878 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445883 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445889 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445897 2572 flags.go:64] FLAG: --tls-min-version="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445901 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445906 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445912 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445916 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445921 2572 flags.go:64] FLAG: --v="2" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445927 2572 flags.go:64] FLAG: --version="false" Jul 11 07:38:30.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445933 2572 flags.go:64] FLAG: --vmodule="" Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445939 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.445944 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446131 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446138 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446143 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446173 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446179 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446184 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446188 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446193 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446198 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446204 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446211 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446216 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446222 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446227 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446232 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446237 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.455033 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446241 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446248 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446254 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446259 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446263 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446270 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446274 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446279 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446283 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446288 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446292 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446297 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446301 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446305 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446309 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446313 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446318 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446322 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446326 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.455484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446330 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446334 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446338 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446342 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446346 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446350 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446354 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446358 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446362 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446366 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446370 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446374 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446378 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446382 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446386 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446390 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446395 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446399 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446405 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446409 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.455984 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446413 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446417 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446421 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446425 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446429 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446433 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446437 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446441 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446446 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446450 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446453 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446457 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446461 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446465 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446469 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446473 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446477 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446481 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446485 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446489 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.456447 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446493 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446497 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446501 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446505 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446509 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446513 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446517 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446521 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446525 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446529 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446535 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446539 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446543 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446547 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446551 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446555 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446559 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446563 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446567 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446571 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.456959 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446575 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446579 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446583 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.446587 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.447205 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.455331 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.455346 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.455365 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.455371 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455428 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455433 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455436 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455440 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455443 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455446 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.457715 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455449 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455453 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455455 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455458 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455461 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455463 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455468 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455471 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455474 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455477 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455480 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455482 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455485 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455488 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455490 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455493 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455498 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455501 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.458150 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455504 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455507 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455510 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455512 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455515 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455518 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455521 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455524 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455527 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455530 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455532 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455535 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455537 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455540 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455542 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455545 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455547 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455550 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455553 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455555 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.458605 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455557 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455560 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455562 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455565 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455568 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455570 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455573 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455575 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455578 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455580 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455583 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455586 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455588 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455591 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455593 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455595 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455598 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455601 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455603 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455606 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.459131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455609 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455611 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455614 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455616 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455619 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455621 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455624 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455626 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455629 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455631 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455634 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455636 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455638 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455641 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455643 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455647 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455649 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455652 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455655 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455657 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.459624 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455659 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455662 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455666 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455669 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455672 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455675 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455677 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455680 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455682 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455685 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455688 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455690 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455693 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455695 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455698 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.460131 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.455720 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455854 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455859 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455862 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455865 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455868 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455870 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455874 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455877 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455879 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455882 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455884 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455887 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455890 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455893 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455895 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455898 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455900 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455903 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455906 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455908 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 07:38:30.460561 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455911 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455913 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455915 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455918 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455920 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455923 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455926 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455928 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455931 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455934 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455937 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455941 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455944 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455947 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455950 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455953 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455955 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455958 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455961 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455963 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 07:38:30.461164 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455966 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455968 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455971 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455974 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455978 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455982 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455985 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455988 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455990 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455993 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455996 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.455998 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456001 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456003 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456006 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456008 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456011 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456013 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456016 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 07:38:30.461652 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456018 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456021 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456023 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456026 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456028 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456030 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456033 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456036 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456040 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456042 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456045 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456047 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456049 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456052 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456054 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456056 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456059 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456061 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456063 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 07:38:30.462193 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456066 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456069 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456072 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456074 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456076 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456079 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456081 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456084 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456086 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456088 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456091 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456093 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456096 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456099 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456101 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456104 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456106 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456109 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456111 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 07:38:30.462664 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456113 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:30.456116 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.456120 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.456239 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.459134 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.460688 2572 server.go:1019] "Starting client certificate rotation" Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.460803 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 11 07:38:30.463175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.460848 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 11 07:38:30.463446 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.463432 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 11 07:38:30.466478 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.466464 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 11 07:38:30.475474 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.475452 2572 log.go:25] "Validated CRI v1 runtime API" Jul 11 07:38:30.478763 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.478746 2572 log.go:25] "Validated CRI v1 image API" Jul 11 07:38:30.479582 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.479565 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 11 07:38:30.482093 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.482074 2572 fs.go:135] Filesystem UUIDs: map[0b735f33-41d7-456b-bca3-5d18129ff850:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f16d8613-389d-4817-8726-8cd2ab73f592:/dev/nvme0n1p3] Jul 11 07:38:30.482142 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.482099 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 11 07:38:30.488113 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.487988 2572 manager.go:216] Machine: {Timestamp:2026-07-11 07:38:30.486547742 +0000 UTC m=+0.332536630 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3202827 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2532cf74356b1aaca620d9cf9e79a7 SystemUUID:ec2532cf-7435-6b1a-aca6-20d9cf9e79a7 BootID:7aee839c-7391-44a6-9d7d-b4274782274a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ac:dd:c7:34:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ac:dd:c7:34:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:66:2d:6e:09:69 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 11 07:38:30.488113 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.488100 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 11 07:38:30.488237 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.488208 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 11 07:38:30.490137 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.490115 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 11 07:38:30.490276 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.490140 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 11 07:38:30.490466 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.490450 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 11 07:38:30.491430 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.491419 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 11 07:38:30.491479 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.491433 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 11 07:38:30.491479 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.491440 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 11 07:38:30.492113 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.492102 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 11 07:38:30.493392 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.493383 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 11 07:38:30.496465 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496454 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 11 07:38:30.496718 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496694 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 11 07:38:30.496843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496820 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 11 07:38:30.496933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496884 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 11 07:38:30.496933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496915 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 11 07:38:30.497043 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496939 2572 kubelet.go:398] "Adding apiserver pod source" Jul 11 07:38:30.497043 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.496956 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 11 07:38:30.497486 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.497469 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 11 07:38:30.497537 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.497499 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 11 07:38:30.500038 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.500016 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 11 07:38:30.501958 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.501943 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 11 07:38:30.502021 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.501974 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 11 07:38:30.502141 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502131 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 11 07:38:30.502210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502146 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 11 07:38:30.502210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502154 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 11 07:38:30.502210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502159 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 11 07:38:30.502210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502176 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 11 07:38:30.502210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502182 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 11 07:38:30.502826 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502816 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 11 07:38:30.502826 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502826 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 11 07:38:30.502900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502833 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 11 07:38:30.502900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502839 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 11 07:38:30.502900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502848 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 11 07:38:30.502900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.502855 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 11 07:38:30.504574 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.504563 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 11 07:38:30.504574 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.504572 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 11 07:38:30.507889 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.507874 2572 server.go:1268] "Started kubelet" Jul 11 07:38:30.508029 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.507959 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 11 07:38:30.508129 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.507973 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 11 07:38:30.508129 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.508046 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 11 07:38:30.508823 ip-10-0-132-10 systemd[1]: Started Kubernetes Kubelet. Jul 11 07:38:30.509509 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.509395 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 11 07:38:30.510139 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.510122 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 11 07:38:30.511668 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.511647 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 11 07:38:30.511792 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.511769 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 11 07:38:30.511941 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.511923 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 11 07:38:30.514828 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.514801 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 11 07:38:30.515280 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.515263 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 11 07:38:30.515362 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.515281 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 11 07:38:30.515444 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.515421 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.516221 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.516200 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 11 07:38:30.516315 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.516201 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 11 07:38:30.516315 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.516237 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 11 07:38:30.516421 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.516338 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 11 07:38:30.516421 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.516348 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 11 07:38:30.517289 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517271 2572 factory.go:153] Registering CRI-O factory Jul 11 07:38:30.517289 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517291 2572 factory.go:223] Registration of the crio container factory successfully Jul 11 07:38:30.517416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517345 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 11 07:38:30.517416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517353 2572 factory.go:55] Registering systemd factory Jul 11 07:38:30.517416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517361 2572 factory.go:223] Registration of the systemd container factory successfully Jul 11 07:38:30.517416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.517384 2572 factory.go:103] Registering Raw factory Jul 11 07:38:30.518062 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.518046 2572 manager.go:1196] Started watching for new ooms in manager Jul 11 07:38:30.518620 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.518606 2572 manager.go:349] Starting recovery of all containers Jul 11 07:38:30.520016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.519992 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-86kgs" Jul 11 07:38:30.523581 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.523550 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 11 07:38:30.523724 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.523698 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 11 07:38:30.524954 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.523718 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-10.ec2.internal.18c12c2fa1f73d3c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-10.ec2.internal,UID:ip-10-0-132-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-10.ec2.internal,},FirstTimestamp:2026-07-11 07:38:30.507846972 +0000 UTC m=+0.353835858,LastTimestamp:2026-07-11 07:38:30.507846972 +0000 UTC m=+0.353835858,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-10.ec2.internal,}" Jul 11 07:38:30.527628 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.527575 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-86kgs" Jul 11 07:38:30.531121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.531004 2572 manager.go:354] Recovery completed Jul 11 07:38:30.535368 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.535354 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.537467 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.537446 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.537533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.537478 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.537533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.537489 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.538040 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538026 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 11 07:38:30.538040 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538039 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 11 07:38:30.538148 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538057 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 11 07:38:30.538148 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538068 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.538741 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538727 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.538808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538754 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.538808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.538764 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.539151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.539138 2572 policy_none.go:49] "None policy: Start" Jul 11 07:38:30.539192 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.539158 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 11 07:38:30.539192 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.539169 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 11 07:38:30.539247 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.539136 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-10.ec2.internal.18c12c2fa3bb2df4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-10.ec2.internal,UID:ip-10-0-132-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-10.ec2.internal,},FirstTimestamp:2026-07-11 07:38:30.537465332 +0000 UTC m=+0.383454221,LastTimestamp:2026-07-11 07:38:30.537465332 +0000 UTC m=+0.383454221,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-10.ec2.internal,}" Jul 11 07:38:30.540431 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.540420 2572 policy_none.go:47] "Start" Jul 11 07:38:30.571945 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.571929 2572 manager.go:337] "Starting Device Plugin manager" Jul 11 07:38:30.572029 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.571968 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 11 07:38:30.572029 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.571981 2572 server.go:87] "Starting device plugin registration server" Jul 11 07:38:30.572242 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.572229 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 11 07:38:30.572300 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.572245 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 11 07:38:30.572346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.572326 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 11 07:38:30.572422 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.572403 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 11 07:38:30.572422 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.572420 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 11 07:38:30.572912 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.572874 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 11 07:38:30.572998 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.572950 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.673040 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.673004 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.675975 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.675957 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.676055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.675990 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.676055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.676000 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.676055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.676032 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.684138 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.684123 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.684190 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.684145 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-10.ec2.internal\": node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.698322 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.698265 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.703252 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.703217 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 11 07:38:30.704476 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.704459 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 11 07:38:30.704571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.704495 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 11 07:38:30.704571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.704517 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 11 07:38:30.704571 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.704549 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 11 07:38:30.706555 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.706539 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 07:38:30.802977 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.802924 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.805264 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.805242 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"] Jul 11 07:38:30.805320 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.805313 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.807074 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.807058 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.807151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.807090 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.807151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.807100 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.808567 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.808554 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.808690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.808674 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.808739 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.808727 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.809446 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809429 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.809446 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809443 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.809585 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809460 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.809585 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809461 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.809585 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809486 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.809585 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.809472 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.811237 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.811220 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.811328 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.811252 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 07:38:30.812073 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.812054 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory" Jul 11 07:38:30.812153 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.812086 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 07:38:30.812153 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.812100 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID" Jul 11 07:38:30.818085 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.818066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.818165 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.818091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749c499dc7c40abc02010a066fc8c1ea-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"749c499dc7c40abc02010a066fc8c1ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.818165 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.818118 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.825352 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.825333 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-10.ec2.internal\" not found" node="ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.829433 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.829415 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-10.ec2.internal\" not found" node="ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.903914 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:30.903869 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:30.918433 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918407 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.918511 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918442 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.918511 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749c499dc7c40abc02010a066fc8c1ea-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"749c499dc7c40abc02010a066fc8c1ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.918596 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.918596 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918529 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749c499dc7c40abc02010a066fc8c1ea-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"749c499dc7c40abc02010a066fc8c1ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:30.918596 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:30.918528 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7d8ca681a96c7d04e9f149226dd3f09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"b7d8ca681a96c7d04e9f149226dd3f09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:31.004418 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.004376 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.105471 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.105422 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.127446 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.127417 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:31.133197 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.133155 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:31.207448 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.207400 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.308112 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.308046 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.409088 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.409053 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.460258 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.460227 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 11 07:38:31.460803 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.460366 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 11 07:38:31.509542 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.509505 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.531923 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.531893 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-10 07:33:30 +0000 UTC" deadline="2027-12-08 17:21:54.749927873 +0000 UTC" Jul 11 07:38:31.531923 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.531918 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12369h43m23.218012824s" Jul 11 07:38:31.609980 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:31.609905 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found" Jul 11 07:38:31.658420 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.658394 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 07:38:31.695072 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.695009 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 07:38:31.715580 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.715543 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" Jul 11 07:38:31.726415 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.726394 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 11 07:38:31.727563 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.727545 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" Jul 11 07:38:31.735896 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.735876 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 11 07:38:31.736948 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:31.736917 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749c499dc7c40abc02010a066fc8c1ea.slice/crio-88b6c5bc15eaf28f96f924e245d17606e61445ba0feb4966b9da093bbfcdad7d WatchSource:0}: Error finding container 88b6c5bc15eaf28f96f924e245d17606e61445ba0feb4966b9da093bbfcdad7d: Status 404 returned error can't find the container with id 88b6c5bc15eaf28f96f924e245d17606e61445ba0feb4966b9da093bbfcdad7d Jul 11 07:38:31.741830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:31.741814 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 07:38:31.746923 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:31.746903 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d8ca681a96c7d04e9f149226dd3f09.slice/crio-609e27736b48cbd95362918f5fca17490c800b5c7716448d8a3a5e98480e8439 WatchSource:0}: Error finding container 609e27736b48cbd95362918f5fca17490c800b5c7716448d8a3a5e98480e8439: Status 404 returned error can't find the container with id 609e27736b48cbd95362918f5fca17490c800b5c7716448d8a3a5e98480e8439 Jul 11 07:38:32.035566 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.035534 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 07:38:32.498999 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.498968 2572 apiserver.go:52] "Watching apiserver" Jul 11 07:38:32.508129 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.508101 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 11 07:38:32.508618 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.508586 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-xll5v","kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5","openshift-cluster-node-tuning-operator/tuned-xgj5w","openshift-image-registry/node-ca-nk6bk","openshift-multus/multus-additional-cni-plugins-4jv2k","openshift-network-diagnostics/network-check-target-8l8cg","kube-system/global-pull-secret-syncer-lhxbc","kube-system/konnectivity-agent-x8plb","openshift-dns/node-resolver-h7j88","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal","openshift-multus/multus-pjrms","openshift-multus/network-metrics-daemon-mcqm7","openshift-network-operator/iptables-alerter-2trw4"] Jul 11 07:38:32.511383 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.511357 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:32.511499 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.511449 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:32.513601 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.513580 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.513715 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.513665 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:32.515870 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.515845 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 11 07:38:32.516605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.516347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.519690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.519281 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zt2cg\"" Jul 11 07:38:32.519690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.519381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.519690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.519519 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 11 07:38:32.519940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.519897 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.520040 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.520006 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.522152 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.521883 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524284 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7h2x7\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524537 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524545 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524550 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524780 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524814 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.524878 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.525554 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.525091 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pzd94\"" Jul 11 07:38:32.527121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.526778 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.527765 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.527732 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 11 07:38:32.528045 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528010 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-sys-fs\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.528131 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528079 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 11 07:38:32.528131 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528111 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bmrdz\"" Jul 11 07:38:32.528932 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528892 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98qq2\" (UniqueName: \"kubernetes.io/projected/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kube-api-access-98qq2\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529024 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528956 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-modprobe-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529024 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.528983 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529130 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529078 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-conf\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529130 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529109 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-sys\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529234 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529139 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-var-lib-kubelet\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529234 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-host\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529234 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529192 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-registration-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529234 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529225 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-device-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-systemd\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-run\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529321 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-lib-modules\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529381 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5glz\" (UniqueName: \"kubernetes.io/projected/feffa75b-07b6-4079-9ed3-7f56aa5897cc-kube-api-access-w5glz\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.529434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529416 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529447 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-socket-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529509 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysconfig\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-kubernetes\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529545 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529560 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529619 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-tuned\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-tmp\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.529730 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.529709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxwxs\" (UniqueName: \"kubernetes.io/projected/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-kube-api-access-dxwxs\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.530171 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.530140 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.530238 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.530154 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 11 07:38:32.530325 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.530305 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.530434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.530306 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tm652\"" Jul 11 07:38:32.530871 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.530851 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.533145 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.533115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.534653 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.534631 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.534866 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.534847 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.535031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.534904 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 11 07:38:32.535175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.535154 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 11 07:38:32.535274 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.535245 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 11 07:38:32.535678 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.535658 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 11 07:38:32.535767 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.535728 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r68tw\"" Jul 11 07:38:32.536107 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.536089 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 11 07:38:32.536894 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.536874 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.537507 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.537359 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbs5g\"" Jul 11 07:38:32.537616 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.537598 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.537779 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.537700 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.538030 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.537938 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 11 07:38:32.539824 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.539793 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.539924 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.539869 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:32.540419 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.540400 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 11 07:38:32.540707 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.540691 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 11 07:38:32.540839 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.540723 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-962mn\"" Jul 11 07:38:32.542373 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.542355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.544950 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.544928 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 11 07:38:32.545093 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.545046 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g7r2d\"" Jul 11 07:38:32.604404 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.604374 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-22qb4" Jul 11 07:38:32.610026 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.610001 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-22qb4" Jul 11 07:38:32.617323 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.617304 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 11 07:38:32.629877 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.629855 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ded7f277-d188-4a8b-b232-80d7fabde531-serviceca\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.630016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.629884 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-kubelet\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.629901 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cni-binary-copy\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.629925 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-os-release\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.630016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.629964 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.630016 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630001 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4kh\" (UniqueName: \"kubernetes.io/projected/b190c905-2852-4d39-a38d-d839596e1bdc-kube-api-access-bj4kh\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndkn\" (UniqueName: \"kubernetes.io/projected/387f703e-cf0c-4e64-a468-cdb16936e32f-kube-api-access-lndkn\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-netns\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630134 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-bin\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630168 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-k8s-cni-cncf-io\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-multus-certs\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlhb\" (UniqueName: \"kubernetes.io/projected/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-kube-api-access-jtlhb\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-systemd\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-run\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630411 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-systemd\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-log-socket\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630471 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-run\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630502 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-netns\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-daemon-config\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b190c905-2852-4d39-a38d-d839596e1bdc-host-slash\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630618 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysconfig\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-kubernetes\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630702 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-systemd-units\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-tuned\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630764 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-kubernetes\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630771 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-tmp\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxwxs\" (UniqueName: \"kubernetes.io/projected/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-kube-api-access-dxwxs\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630830 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-systemd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630775 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysconfig\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.630862 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630855 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-etc-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630933 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.630973 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-kubelet\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-conf-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631041 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98qq2\" (UniqueName: \"kubernetes.io/projected/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kube-api-access-98qq2\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631085 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-sys\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631092 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631160 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631192 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-sys\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631239 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns86m\" (UniqueName: \"kubernetes.io/projected/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-kube-api-access-ns86m\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631260 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-socket-dir-parent\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631305 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631355 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631372 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-netd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.631425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-config\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-hostroot\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631526 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-slash\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631823 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-etc-kubernetes\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631876 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.632144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.631955 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70fd8587-ba55-4187-a314-4fa49e388105-hosts-file\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.632457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h72nk\" (UniqueName: \"kubernetes.io/projected/70fd8587-ba55-4187-a314-4fa49e388105-kube-api-access-h72nk\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.632457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632197 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.632457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632233 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-modprobe-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632944 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-var-lib-kubelet\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-host\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632987 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-var-lib-kubelet\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632983 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633031 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.632945 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-modprobe-d\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633042 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633094 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-bin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633044 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-host\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-multus\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8rgj\" (UniqueName: \"kubernetes.io/projected/ded7f277-d188-4a8b-b232-80d7fabde531-kube-api-access-k8rgj\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633177 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633200 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b190c905-2852-4d39-a38d-d839596e1bdc-iptables-alerter-script\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-kubelet-config\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.633351 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cnibin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633377 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fd8587-ba55-4187-a314-4fa49e388105-tmp-dir\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.633441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633423 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-system-cni-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.633460 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:38:33.133424072 +0000 UTC m=+2.979412969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633551 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27e480f2-4a52-462f-be78-567706460221-agent-certs\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633580 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-node-log\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633625 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-env-overrides\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-os-release\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633689 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-sys-fs\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-conf\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ded7f277-d188-4a8b-b232-80d7fabde531-host\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633774 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-cnibin\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633792 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-sys-fs\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633799 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-ovn\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-script-lib\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633861 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-system-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-registration-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633888 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-sysctl-conf\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.634155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633909 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-device-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633939 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-lib-modules\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633956 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-registration-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633965 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5glz\" (UniqueName: \"kubernetes.io/projected/feffa75b-07b6-4079-9ed3-7f56aa5897cc-kube-api-access-w5glz\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.633999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27e480f2-4a52-462f-be78-567706460221-konnectivity-ca\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634025 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-var-lib-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-dbus\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634082 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-socket-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634206 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-device-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634301 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634323 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02c9ed56-b51c-40ea-8799-82c40e6ef77c-socket-dir\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.634879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.634415 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-lib-modules\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.635421 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.635252 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-etc-tuned\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.635421 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.635308 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-tmp\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.637599 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.637576 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:32.637599 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.637601 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:32.637843 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.637616 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:32.637843 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.637760 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:38:33.137678424 +0000 UTC m=+2.983667320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:32.639740 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.639683 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98qq2\" (UniqueName: \"kubernetes.io/projected/02c9ed56-b51c-40ea-8799-82c40e6ef77c-kube-api-access-98qq2\") pod \"aws-ebs-csi-driver-node-nvwz5\" (UID: \"02c9ed56-b51c-40ea-8799-82c40e6ef77c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.640704 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.640674 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxwxs\" (UniqueName: \"kubernetes.io/projected/ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b-kube-api-access-dxwxs\") pod \"tuned-xgj5w\" (UID: \"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b\") " pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.644070 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.644050 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5glz\" (UniqueName: \"kubernetes.io/projected/feffa75b-07b6-4079-9ed3-7f56aa5897cc-kube-api-access-w5glz\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:32.709717 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.709646 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"b7d8ca681a96c7d04e9f149226dd3f09","Type":"ContainerStarted","Data":"609e27736b48cbd95362918f5fca17490c800b5c7716448d8a3a5e98480e8439"} Jul 11 07:38:32.710936 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.710905 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" event={"ID":"749c499dc7c40abc02010a066fc8c1ea","Type":"ContainerStarted","Data":"88b6c5bc15eaf28f96f924e245d17606e61445ba0feb4966b9da093bbfcdad7d"} Jul 11 07:38:32.735201 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735173 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-multus\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.735346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8rgj\" (UniqueName: \"kubernetes.io/projected/ded7f277-d188-4a8b-b232-80d7fabde531-kube-api-access-k8rgj\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.735346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.735346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735253 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b190c905-2852-4d39-a38d-d839596e1bdc-iptables-alerter-script\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.735495 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735384 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-multus\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.735543 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735523 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-kubelet-config\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.735595 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735569 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cnibin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.735640 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735590 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fd8587-ba55-4187-a314-4fa49e388105-tmp-dir\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.735640 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735632 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-system-cni-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735658 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27e480f2-4a52-462f-be78-567706460221-agent-certs\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735669 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-kubelet-config\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735684 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-node-log\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735695 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cnibin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-env-overrides\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.735732 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735745 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-system-cni-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-os-release\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735775 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-node-log\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ded7f277-d188-4a8b-b232-80d7fabde531-host\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735842 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-os-release\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735863 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-cnibin\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735883 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ded7f277-d188-4a8b-b232-80d7fabde531-host\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-ovn\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-script-lib\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735923 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-cnibin\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735943 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-system-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735950 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b190c905-2852-4d39-a38d-d839596e1bdc-iptables-alerter-script\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735968 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-ovn\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27e480f2-4a52-462f-be78-567706460221-konnectivity-ca\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.735998 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-var-lib-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-dbus\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736019 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-system-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.736050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ded7f277-d188-4a8b-b232-80d7fabde531-serviceca\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736075 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-kubelet\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736076 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-var-lib-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736099 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cni-binary-copy\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736108 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fd8587-ba55-4187-a314-4fa49e388105-tmp-dir\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-os-release\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736193 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-os-release\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736206 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4kh\" (UniqueName: \"kubernetes.io/projected/b190c905-2852-4d39-a38d-d839596e1bdc-kube-api-access-bj4kh\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndkn\" (UniqueName: \"kubernetes.io/projected/387f703e-cf0c-4e64-a468-cdb16936e32f-kube-api-access-lndkn\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736242 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-kubelet\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-netns\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736285 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-env-overrides\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736339 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736503 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-bin\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736528 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.737230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736555 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-k8s-cni-cncf-io\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736572 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ded7f277-d188-4a8b-b232-80d7fabde531-serviceca\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736574 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-script-lib\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736595 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-run-netns\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-multus-certs\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736615 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-multus-certs\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736599 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27e480f2-4a52-462f-be78-567706460221-konnectivity-ca\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736625 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21b00770-91a4-47b6-ae61-6eacb490ae55-dbus\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736645 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlhb\" (UniqueName: \"kubernetes.io/projected/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-kube-api-access-jtlhb\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736674 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-bin\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736694 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-cni-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736707 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-k8s-cni-cncf-io\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736715 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-log-socket\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-netns\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-log-socket\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736866 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-daemon-config\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736954 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.738068 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.736985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b190c905-2852-4d39-a38d-d839596e1bdc-host-slash\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737010 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-run-netns\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737071 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b190c905-2852-4d39-a38d-d839596e1bdc-host-slash\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-systemd-units\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737140 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-systemd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-etc-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737187 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737190 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-systemd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737195 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-cni-binary-copy\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737213 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-run-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737218 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-etc-openvswitch\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737168 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-systemd-units\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-kubelet\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737253 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-kubelet\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-conf-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737307 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737333 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns86m\" (UniqueName: \"kubernetes.io/projected/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-kube-api-access-ns86m\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737337 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-daemon-config\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.738904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737361 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-conf-dir\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737360 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-socket-dir-parent\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737415 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-multus-socket-dir-parent\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737421 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-netd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-cni-netd\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737452 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-config\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737470 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/387f703e-cf0c-4e64-a468-cdb16936e32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-hostroot\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.737511 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-slash\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:32.737559 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:33.237540187 +0000 UTC m=+3.083529062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737567 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-slash\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-etc-kubernetes\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737583 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-hostroot\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70fd8587-ba55-4187-a314-4fa49e388105-hosts-file\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h72nk\" (UniqueName: \"kubernetes.io/projected/70fd8587-ba55-4187-a314-4fa49e388105-kube-api-access-h72nk\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.739702 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737621 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-etc-kubernetes\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737656 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737667 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70fd8587-ba55-4187-a314-4fa49e388105-hosts-file\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737743 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-bin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737877 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovnkube-config\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.737984 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-host-var-lib-cni-bin\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.738271 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/387f703e-cf0c-4e64-a468-cdb16936e32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.740491 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.738935 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27e480f2-4a52-462f-be78-567706460221-agent-certs\") pod \"konnectivity-agent-x8plb\" (UID: \"27e480f2-4a52-462f-be78-567706460221\") " pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.740927 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.740549 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.746023 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.745999 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8rgj\" (UniqueName: \"kubernetes.io/projected/ded7f277-d188-4a8b-b232-80d7fabde531-kube-api-access-k8rgj\") pod \"node-ca-nk6bk\" (UID: \"ded7f277-d188-4a8b-b232-80d7fabde531\") " pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.747151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.747126 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndkn\" (UniqueName: \"kubernetes.io/projected/387f703e-cf0c-4e64-a468-cdb16936e32f-kube-api-access-lndkn\") pod \"multus-additional-cni-plugins-4jv2k\" (UID: \"387f703e-cf0c-4e64-a468-cdb16936e32f\") " pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.747249 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.747230 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4kh\" (UniqueName: \"kubernetes.io/projected/b190c905-2852-4d39-a38d-d839596e1bdc-kube-api-access-bj4kh\") pod \"iptables-alerter-2trw4\" (UID: \"b190c905-2852-4d39-a38d-d839596e1bdc\") " pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.751640 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.751566 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns86m\" (UniqueName: \"kubernetes.io/projected/bc85b57d-9265-4576-a91e-5d75fa1a0d0b-kube-api-access-ns86m\") pod \"ovnkube-node-xll5v\" (UID: \"bc85b57d-9265-4576-a91e-5d75fa1a0d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.752297 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.752279 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h72nk\" (UniqueName: \"kubernetes.io/projected/70fd8587-ba55-4187-a314-4fa49e388105-kube-api-access-h72nk\") pod \"node-resolver-h7j88\" (UID: \"70fd8587-ba55-4187-a314-4fa49e388105\") " pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.765362 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.765336 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlhb\" (UniqueName: \"kubernetes.io/projected/2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce-kube-api-access-jtlhb\") pod \"multus-pjrms\" (UID: \"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce\") " pod="openshift-multus/multus-pjrms" Jul 11 07:38:32.833756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.833707 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" Jul 11 07:38:32.842612 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.842586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" Jul 11 07:38:32.855685 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.855655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2trw4" Jul 11 07:38:32.861490 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.861465 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nk6bk" Jul 11 07:38:32.869285 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.869266 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" Jul 11 07:38:32.878146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.878125 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:32.886115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.886084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:32.900818 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.900796 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h7j88" Jul 11 07:38:32.905903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:32.905882 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pjrms" Jul 11 07:38:33.049628 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.049549 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 07:38:33.141395 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.141358 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:33.141559 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.141414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:33.141559 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141526 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:33.141635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141555 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:33.141635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141581 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:33.141635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141584 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:38:34.141565341 +0000 UTC m=+3.987554215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:33.141635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141595 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:33.141807 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.141648 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:38:34.14163094 +0000 UTC m=+3.987619816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:33.242684 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.242646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:33.242872 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.242832 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:33.242950 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.242910 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:34.242888134 +0000 UTC m=+4.088877011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:33.613945 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.613898 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 07:33:32 +0000 UTC" deadline="2028-03-05 09:57:34.637180864 +0000 UTC" Jul 11 07:38:33.613945 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.613940 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14474h19m1.023246509s" Jul 11 07:38:33.705254 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.705220 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:33.705364 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:33.705220 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:33.705364 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.705333 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:33.705482 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:33.705441 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:33.713947 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.713924 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb190c905_2852_4d39_a38d_d839596e1bdc.slice/crio-9080839d4f7110f2f489ec265401823c500b4b51100d76c776c85e30a3f80eb7 WatchSource:0}: Error finding container 9080839d4f7110f2f489ec265401823c500b4b51100d76c776c85e30a3f80eb7: Status 404 returned error can't find the container with id 9080839d4f7110f2f489ec265401823c500b4b51100d76c776c85e30a3f80eb7 Jul 11 07:38:33.715404 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.715378 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fd8587_ba55_4187_a314_4fa49e388105.slice/crio-6791e598bb74111b61e05aab74f814f565f8a98bad9c7edd50ccd29724ec15fc WatchSource:0}: Error finding container 6791e598bb74111b61e05aab74f814f565f8a98bad9c7edd50ccd29724ec15fc: Status 404 returned error can't find the container with id 6791e598bb74111b61e05aab74f814f565f8a98bad9c7edd50ccd29724ec15fc Jul 11 07:38:33.716049 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.716010 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e480f2_4a52_462f_be78_567706460221.slice/crio-6af84ecfd6a86cf8af084d2137a5b0a40a55488b3f653797ab74cc5edc7a2937 WatchSource:0}: Error finding container 6af84ecfd6a86cf8af084d2137a5b0a40a55488b3f653797ab74cc5edc7a2937: Status 404 returned error can't find the container with id 6af84ecfd6a86cf8af084d2137a5b0a40a55488b3f653797ab74cc5edc7a2937 Jul 11 07:38:33.723923 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.723894 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d67ce78_ea2a_47a4_b1f9_58c5d6e786ce.slice/crio-ac921b9e52faa8263d8f506483b992eb943d444f378327d9f75ab7848a8bf4ac WatchSource:0}: Error finding container ac921b9e52faa8263d8f506483b992eb943d444f378327d9f75ab7848a8bf4ac: Status 404 returned error can't find the container with id ac921b9e52faa8263d8f506483b992eb943d444f378327d9f75ab7848a8bf4ac Jul 11 07:38:33.724589 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.724554 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded7f277_d188_4a8b_b232_80d7fabde531.slice/crio-d9322d868f1c8c7dc493038a6d7c2e20de269d15f63fb97f513ce01f0dbe049d WatchSource:0}: Error finding container d9322d868f1c8c7dc493038a6d7c2e20de269d15f63fb97f513ce01f0dbe049d: Status 404 returned error can't find the container with id d9322d868f1c8c7dc493038a6d7c2e20de269d15f63fb97f513ce01f0dbe049d Jul 11 07:38:33.725928 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.725901 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc85b57d_9265_4576_a91e_5d75fa1a0d0b.slice/crio-4f11f46ba49bd301ff4b8fd7a3409b64f2dd3ac3467fcefac91914a4f243a0a7 WatchSource:0}: Error finding container 4f11f46ba49bd301ff4b8fd7a3409b64f2dd3ac3467fcefac91914a4f243a0a7: Status 404 returned error can't find the container with id 4f11f46ba49bd301ff4b8fd7a3409b64f2dd3ac3467fcefac91914a4f243a0a7 Jul 11 07:38:33.727483 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.727431 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc28799_7f41_4d5c_b1e3_d58af0ba9d0b.slice/crio-3f4dab89d642e58b660f5fab087a353dfd72575bd8fe7a3e0db3274fa2507fe0 WatchSource:0}: Error finding container 3f4dab89d642e58b660f5fab087a353dfd72575bd8fe7a3e0db3274fa2507fe0: Status 404 returned error can't find the container with id 3f4dab89d642e58b660f5fab087a353dfd72575bd8fe7a3e0db3274fa2507fe0 Jul 11 07:38:33.728972 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:38:33.728861 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387f703e_cf0c_4e64_a468_cdb16936e32f.slice/crio-05b375d25f6c2311f9327abeb90b30a9cc5c3473b813c9f77b3263c6b11382bb WatchSource:0}: Error finding container 05b375d25f6c2311f9327abeb90b30a9cc5c3473b813c9f77b3263c6b11382bb: Status 404 returned error can't find the container with id 05b375d25f6c2311f9327abeb90b30a9cc5c3473b813c9f77b3263c6b11382bb Jul 11 07:38:34.150727 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.150378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:34.150727 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.150693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:34.150975 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.150886 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:34.150975 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.150957 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:38:36.150935185 +0000 UTC m=+5.996924063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:34.151180 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.151163 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:34.151250 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.151186 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:34.151250 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.151199 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:34.151250 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.151241 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:38:36.151228019 +0000 UTC m=+5.997216902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:34.251169 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.251106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:34.251349 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.251302 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:34.251410 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.251367 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:36.251348136 +0000 UTC m=+6.097337016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:34.614181 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.614113 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 07:33:32 +0000 UTC" deadline="2028-02-18 21:25:07.275356217 +0000 UTC" Jul 11 07:38:34.614181 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.614179 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14101h46m32.661181463s" Jul 11 07:38:34.707071 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.707030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:34.707259 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:34.707202 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:34.729375 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.729333 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h7j88" event={"ID":"70fd8587-ba55-4187-a314-4fa49e388105","Type":"ContainerStarted","Data":"6791e598bb74111b61e05aab74f814f565f8a98bad9c7edd50ccd29724ec15fc"} Jul 11 07:38:34.737282 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.737252 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2trw4" event={"ID":"b190c905-2852-4d39-a38d-d839596e1bdc","Type":"ContainerStarted","Data":"9080839d4f7110f2f489ec265401823c500b4b51100d76c776c85e30a3f80eb7"} Jul 11 07:38:34.752479 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.752449 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nk6bk" event={"ID":"ded7f277-d188-4a8b-b232-80d7fabde531","Type":"ContainerStarted","Data":"d9322d868f1c8c7dc493038a6d7c2e20de269d15f63fb97f513ce01f0dbe049d"} Jul 11 07:38:34.756601 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.756563 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"4f11f46ba49bd301ff4b8fd7a3409b64f2dd3ac3467fcefac91914a4f243a0a7"} Jul 11 07:38:34.768076 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.768025 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pjrms" event={"ID":"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce","Type":"ContainerStarted","Data":"ac921b9e52faa8263d8f506483b992eb943d444f378327d9f75ab7848a8bf4ac"} Jul 11 07:38:34.781334 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.781302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x8plb" event={"ID":"27e480f2-4a52-462f-be78-567706460221","Type":"ContainerStarted","Data":"6af84ecfd6a86cf8af084d2137a5b0a40a55488b3f653797ab74cc5edc7a2937"} Jul 11 07:38:34.795055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.795019 2572 generic.go:356] "Generic (PLEG): container finished" podID="b7d8ca681a96c7d04e9f149226dd3f09" containerID="99b96f7b3fde7bf75673cc3cb73a263aad836a7323095ed671e245403d66e526" exitCode=0 Jul 11 07:38:34.795184 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.795105 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"b7d8ca681a96c7d04e9f149226dd3f09","Type":"ContainerDied","Data":"99b96f7b3fde7bf75673cc3cb73a263aad836a7323095ed671e245403d66e526"} Jul 11 07:38:34.811348 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.810892 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" event={"ID":"749c499dc7c40abc02010a066fc8c1ea","Type":"ContainerStarted","Data":"a48984c425b7efbf1869d8d3a83fbf5bcbddc365788d0aaaf80c2b1da4cbffe0"} Jul 11 07:38:34.824841 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.824812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerStarted","Data":"05b375d25f6c2311f9327abeb90b30a9cc5c3473b813c9f77b3263c6b11382bb"} Jul 11 07:38:34.841019 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.840988 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" event={"ID":"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b","Type":"ContainerStarted","Data":"3f4dab89d642e58b660f5fab087a353dfd72575bd8fe7a3e0db3274fa2507fe0"} Jul 11 07:38:34.861802 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:34.861756 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" event={"ID":"02c9ed56-b51c-40ea-8799-82c40e6ef77c","Type":"ContainerStarted","Data":"d2948aa8d88c5a41b1e5c8fa4fc44f253b0151ddf0eb5ae6b2ba8fdc1d05b267"} Jul 11 07:38:35.705260 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:35.705219 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:35.705809 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:35.705383 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:35.705809 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:35.705486 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:35.705809 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:35.705564 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:35.882919 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:35.882847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"b7d8ca681a96c7d04e9f149226dd3f09","Type":"ContainerStarted","Data":"ea1cbfedde14ec3e688105f94fb8730448333c36e88730aed8b612fa83f066b2"} Jul 11 07:38:35.901380 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:35.901310 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" podStartSLOduration=3.901288553 podStartE2EDuration="3.901288553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:34.824791761 +0000 UTC m=+4.670780657" watchObservedRunningTime="2026-07-11 07:38:35.901288553 +0000 UTC m=+5.747277450" Jul 11 07:38:36.172673 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:36.172623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:36.172886 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:36.172690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:36.172951 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.172884 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:36.172951 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.172951 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:38:40.172929936 +0000 UTC m=+10.018918816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:36.173075 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.173044 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:36.173075 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.173059 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:36.173075 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.173073 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:36.173223 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.173108 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:38:40.173096573 +0000 UTC m=+10.019085452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:36.273545 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:36.273476 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:36.273745 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.273690 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:36.273891 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.273842 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:40.273817206 +0000 UTC m=+10.119806094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:36.706322 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:36.706263 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:36.706830 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:36.706419 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:37.705955 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:37.705689 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:37.705955 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:37.705839 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:37.705955 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:37.705896 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:37.706238 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:37.706011 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:38.705888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:38.705791 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:38.706319 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:38.705957 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:39.705041 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:39.705003 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:39.705224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:39.705058 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:39.705224 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:39.705149 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:39.705342 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:39.705253 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:40.207543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:40.207627 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.207777 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.207796 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.207809 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.207863 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:38:48.207845961 +0000 UTC m=+18.053834840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:40.208183 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.208174 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:40.208864 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.208225 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:38:48.208210037 +0000 UTC m=+18.054198920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:40.308533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:40.307996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:40.308533 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.308162 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:40.308533 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.308216 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:38:48.308200157 +0000 UTC m=+18.154189030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:40.706857 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:40.706826 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:40.707046 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:40.706952 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:41.705136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:41.705056 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:41.705136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:41.705088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:41.705630 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:41.705190 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:41.705688 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:41.705619 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:42.705087 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:42.704974 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:42.705254 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:42.705120 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:43.704974 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:43.704938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:43.705154 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:43.704938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:43.705154 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:43.705067 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:43.705154 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:43.705111 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:44.705045 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:44.705016 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:44.705226 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:44.705149 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:45.705111 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:45.705080 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:45.705571 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:45.705205 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:45.705571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:45.705271 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:45.705571 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:45.705390 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:46.705588 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:46.705545 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:46.705977 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:46.705669 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:47.705032 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:47.704997 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:47.705210 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:47.704997 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:47.705210 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:47.705103 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:47.705210 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:47.705165 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:48.274718 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:48.274677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:48.274745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274862 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274874 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274897 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274907 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274924 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:39:04.274908266 +0000 UTC m=+34.120897137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:38:48.275169 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.274950 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:39:04.274933487 +0000 UTC m=+34.120922377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:38:48.375652 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:48.375618 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:48.375832 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.375757 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:48.375832 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.375811 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:04.375797718 +0000 UTC m=+34.221786590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:38:48.705002 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:48.704914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:48.705171 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:48.705041 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:49.705061 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:49.705030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:49.705451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:49.705030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:49.705451 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:49.705165 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:49.705451 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:49.705231 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:50.706296 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:50.706260 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:50.706696 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:50.706388 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:51.704876 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:51.704836 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:51.705055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:51.704846 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:51.705055 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:51.704984 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:51.705055 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:51.705027 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:52.705200 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.704921 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:52.705999 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:52.705594 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:52.918906 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.918880 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" event={"ID":"ddc28799-7f41-4d5c-b1e3-d58af0ba9d0b","Type":"ContainerStarted","Data":"2aa285c4947e7c0b6e342db3b00c53ef39a1924442da3cc0e4e6a49bd13efda8"} Jul 11 07:38:52.920443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.920418 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" event={"ID":"02c9ed56-b51c-40ea-8799-82c40e6ef77c","Type":"ContainerStarted","Data":"0d911b38807af3c3f0187f74852637696a2bb2036e1ace77af22f64c627fba08"} Jul 11 07:38:52.921848 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.921820 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h7j88" event={"ID":"70fd8587-ba55-4187-a314-4fa49e388105","Type":"ContainerStarted","Data":"fea0b978a7298d394c7c1e8b4bba2e78784ad38a6eaeb392d8a08624b555d154"} Jul 11 07:38:52.923465 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.923439 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nk6bk" event={"ID":"ded7f277-d188-4a8b-b232-80d7fabde531","Type":"ContainerStarted","Data":"6a47e3c0ebb93740039a6cfa342f09ca9adb4c6b89223ab5cea88e240d7b7677"} Jul 11 07:38:52.925813 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.925787 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"3c1ced6113cb89d050e8db9f2c8622c75b2e6a7e31b8113340e5b19b6b56d9b7"} Jul 11 07:38:52.925912 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.925836 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"6974aa05dd6e0b29faaa168c886ee9122d5fab5ab77f68a4f721cec737e711ff"} Jul 11 07:38:52.925912 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.925882 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"35608cb1bfb96ca45b8848ae0d51a09867fd29698db606114b85a04f93528a22"} Jul 11 07:38:52.925991 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.925912 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"4e004b2eeb1d7699256d6a60563874bd38712d39f6b11cd23a4899ea05bd3267"} Jul 11 07:38:52.925991 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.925945 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"5dafb44c4016bc9a75c856e8349c762445cd2e1cc8fccb103ccf1adbd682a29c"} Jul 11 07:38:52.927065 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.927047 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pjrms" event={"ID":"2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce","Type":"ContainerStarted","Data":"32e810ba1104c4ba95d1a555b60679d3d244752f5066b1143fd338f50b0121aa"} Jul 11 07:38:52.928256 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.928239 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x8plb" event={"ID":"27e480f2-4a52-462f-be78-567706460221","Type":"ContainerStarted","Data":"418c625996b59b83f87106646f803071cd1cb21c8bbfad109b3c1776f607164e"} Jul 11 07:38:52.929696 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.929673 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="977034075c42146287b6128f12a42d38f6d056c330d5a20b40f0c5d2d463276b" exitCode=0 Jul 11 07:38:52.929796 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.929721 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"977034075c42146287b6128f12a42d38f6d056c330d5a20b40f0c5d2d463276b"} Jul 11 07:38:52.935025 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.934988 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" podStartSLOduration=20.934976715 podStartE2EDuration="20.934976715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:35.902080282 +0000 UTC m=+5.748069178" watchObservedRunningTime="2026-07-11 07:38:52.934976715 +0000 UTC m=+22.780965607" Jul 11 07:38:52.935407 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.935381 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xgj5w" podStartSLOduration=3.338341191 podStartE2EDuration="21.935371154s" totalImagesPullingTime="18.597029963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.934797061 +0000 UTC m=+22.780785953" watchObservedRunningTime="2026-07-11 07:38:52.935371154 +0000 UTC m=+22.781360046" Jul 11 07:38:52.948748 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.948671 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nk6bk" podStartSLOduration=3.352937805 podStartE2EDuration="21.948653461s" totalImagesPullingTime="18.595715656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.947733089 +0000 UTC m=+22.793722011" watchObservedRunningTime="2026-07-11 07:38:52.948653461 +0000 UTC m=+22.794642355" Jul 11 07:38:52.982263 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.982211 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x8plb" podStartSLOduration=11.550101149 podStartE2EDuration="21.982193089s" totalImagesPullingTime="10.43209194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.981532167 +0000 UTC m=+22.827521061" watchObservedRunningTime="2026-07-11 07:38:52.982193089 +0000 UTC m=+22.828181984" Jul 11 07:38:52.995557 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:52.995513 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h7j88" podStartSLOduration=3.389359586 podStartE2EDuration="21.995499441s" totalImagesPullingTime="18.606139855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:52.995172628 +0000 UTC m=+22.841161521" watchObservedRunningTime="2026-07-11 07:38:52.995499441 +0000 UTC m=+22.841488334" Jul 11 07:38:53.012204 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.012155 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pjrms" podStartSLOduration=3.365407383 podStartE2EDuration="22.012138093s" totalImagesPullingTime="18.64673071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:53.012096016 +0000 UTC m=+22.858084912" watchObservedRunningTime="2026-07-11 07:38:53.012138093 +0000 UTC m=+22.858126986" Jul 11 07:38:53.705420 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.705395 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:53.705758 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:53.705524 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:53.705758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.705552 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:53.705758 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:53.705615 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:53.842664 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.842640 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 11 07:38:53.933775 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.933677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" event={"ID":"02c9ed56-b51c-40ea-8799-82c40e6ef77c","Type":"ContainerStarted","Data":"a55818e64598d4bff35536cac989f77c7a7ad02e46cc2fd310ebcc82b85d1b01"} Jul 11 07:38:53.935321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.935290 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2trw4" event={"ID":"b190c905-2852-4d39-a38d-d839596e1bdc","Type":"ContainerStarted","Data":"acf1bcb5b8c5645dd0447be25e0b5ecaafe64c9af122f8a10a41d60ccaef868e"} Jul 11 07:38:53.938755 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.938726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"cf5bda91f3258e5ef34da7d8414a01af998e7d2109b9ddfc0b4469876f20c167"} Jul 11 07:38:53.949920 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:53.949875 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2trw4" podStartSLOduration=4.341827208 podStartE2EDuration="22.949862982s" totalImagesPullingTime="18.608035774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:53.949767767 +0000 UTC m=+23.795756661" watchObservedRunningTime="2026-07-11 07:38:53.949862982 +0000 UTC m=+23.795851874" Jul 11 07:38:54.587424 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:54.587319 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-11T07:38:53.842660725Z","UUID":"0ad19842-223c-4666-9490-f20cc1bffb40","Handler":null,"Name":"","Endpoint":""} Jul 11 07:38:54.588921 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:54.588899 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 11 07:38:54.589049 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:54.588942 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 11 07:38:54.705475 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:54.705430 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:54.705937 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:54.705580 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:55.705249 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:55.705215 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:55.705398 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:55.705339 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:55.705441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:55.705391 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:55.705500 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:55.705482 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:55.945923 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:55.945893 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"7343fac70224e97dfa288d9405e8a3e7af3c9ac0f030bcb6e0017b38f6cb6dc1"} Jul 11 07:38:55.948083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:55.948055 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" event={"ID":"02c9ed56-b51c-40ea-8799-82c40e6ef77c","Type":"ContainerStarted","Data":"6df163e750212b3c562aaa7b548ed7db07b922b26860527047c6b4d79c30d197"} Jul 11 07:38:55.964118 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:55.964048 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvwz5" podStartSLOduration=4.024845982 podStartE2EDuration="24.964035422s" totalImagesPullingTime="20.93918944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:55.963755584 +0000 UTC m=+25.809744477" watchObservedRunningTime="2026-07-11 07:38:55.964035422 +0000 UTC m=+25.810024314" Jul 11 07:38:56.705750 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:56.705460 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:56.706166 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:56.705884 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:57.705217 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.705003 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:57.705313 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.705030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:57.705313 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:57.705262 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:57.705390 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:57.705326 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:57.798604 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.798580 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:57.799343 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.799092 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:57.954448 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.954421 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="f7248653a0f256ba2b7f6473173d097d3565263adf327d775107f6f964e32d41" exitCode=0 Jul 11 07:38:57.954584 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.954500 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"f7248653a0f256ba2b7f6473173d097d3565263adf327d775107f6f964e32d41"} Jul 11 07:38:57.958292 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.958270 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" event={"ID":"bc85b57d-9265-4576-a91e-5d75fa1a0d0b","Type":"ContainerStarted","Data":"ecd0e7b5c7f87084e4471fc3d521fb9ed940fec90c014559e8398410fa42867a"} Jul 11 07:38:57.999117 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:57.999079 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" podStartSLOduration=8.335051911 podStartE2EDuration="26.999067472s" totalImagesPullingTime="18.664015561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:38:57.997485698 +0000 UTC m=+27.843474590" watchObservedRunningTime="2026-07-11 07:38:57.999067472 +0000 UTC m=+27.845056365" Jul 11 07:38:58.262086 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.262059 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:58.262535 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.262518 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x8plb" Jul 11 07:38:58.705487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.705408 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:58.705919 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:58.705569 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:58.960444 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.960377 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:58.960849 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.960465 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:58.960849 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.960520 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:58.975322 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.975296 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:58.975944 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:58.975921 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:38:59.239686 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.239654 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lhxbc"] Jul 11 07:38:59.239800 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.239773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:38:59.239873 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:59.239856 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:38:59.242313 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.242290 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l8cg"] Jul 11 07:38:59.242409 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.242396 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:38:59.242518 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:59.242499 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:38:59.242929 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.242911 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mcqm7"] Jul 11 07:38:59.243007 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.242986 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:38:59.243091 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:38:59.243077 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:38:59.963844 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.963810 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="919fb6b14e837835469ccd0ea067de8af51c9ae435522a08d89e72cb0ab43124" exitCode=0 Jul 11 07:38:59.964287 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:38:59.963890 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"919fb6b14e837835469ccd0ea067de8af51c9ae435522a08d89e72cb0ab43124"} Jul 11 07:39:00.705830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:00.705796 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:00.705987 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:00.705906 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:00.705987 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:00.705900 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:39:00.705987 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:00.705973 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:39:00.706108 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:00.706022 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:00.706155 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:00.706130 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:39:01.970041 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:01.970008 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="a4d55f120fad7f531d38ed8446d4de4e8d8b9d23a5792a6844596e3bbc181d72" exitCode=0 Jul 11 07:39:01.970375 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:01.970082 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"a4d55f120fad7f531d38ed8446d4de4e8d8b9d23a5792a6844596e3bbc181d72"} Jul 11 07:39:02.705634 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:02.705598 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:02.705827 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:02.705604 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:02.705827 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:02.705731 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:39:02.705827 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:02.705752 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:02.705995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:02.705868 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:39:02.705995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:02.705941 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:39:04.306429 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.306397 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.306477 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306537 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306606 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306614 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:39:36.306592448 +0000 UTC m=+66.152581336 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306623 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306634 2572 projected.go:196] Error preparing data for projected volume kube-api-access-bk5hr for pod openshift-network-diagnostics/network-check-target-8l8cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:39:04.306995 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.306677 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr podName:8bb45ac4-39ba-4414-bdb0-286454433faf nodeName:}" failed. No retries permitted until 2026-07-11 07:39:36.306664365 +0000 UTC m=+66.152653250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk5hr" (UniqueName: "kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr") pod "network-check-target-8l8cg" (UID: "8bb45ac4-39ba-4414-bdb0-286454433faf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 07:39:04.407748 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.407694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:04.407896 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.407834 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 07:39:04.407949 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.407904 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret podName:21b00770-91a4-47b6-ae61-6eacb490ae55 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:36.407885326 +0000 UTC m=+66.253874236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret") pod "global-pull-secret-syncer-lhxbc" (UID: "21b00770-91a4-47b6-ae61-6eacb490ae55") : object "kube-system"/"original-pull-secret" not registered Jul 11 07:39:04.705377 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.705301 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:04.705377 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.705316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:04.705585 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.705424 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l8cg" podUID="8bb45ac4-39ba-4414-bdb0-286454433faf" Jul 11 07:39:04.705585 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.705540 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:39:04.705680 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.705596 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:04.705680 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:04.705661 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lhxbc" podUID="21b00770-91a4-47b6-ae61-6eacb490ae55" Jul 11 07:39:04.986378 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.986350 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeReady" Jul 11 07:39:04.986549 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:04.986507 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 11 07:39:05.027897 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.027869 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qzl8f"] Jul 11 07:39:05.059661 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.059630 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6wnzk"] Jul 11 07:39:05.059828 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.059784 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.062858 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.062828 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 11 07:39:05.062975 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.062860 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbf8m\"" Jul 11 07:39:05.062975 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.062834 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 11 07:39:05.073904 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.073883 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qzl8f"] Jul 11 07:39:05.074005 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.073924 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6wnzk"] Jul 11 07:39:05.074005 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.074003 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.077625 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.077575 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 11 07:39:05.077940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.077921 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 11 07:39:05.078035 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.077949 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 11 07:39:05.078198 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.078180 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-57t4x\"" Jul 11 07:39:05.113583 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113551 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27009180-8547-4131-8824-d912c8fe24a3-tmp-dir\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.113718 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113596 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-cert\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.113718 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqnzm\" (UniqueName: \"kubernetes.io/projected/27009180-8547-4131-8824-d912c8fe24a3-kube-api-access-zqnzm\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.113834 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113729 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.113834 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113766 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjd4\" (UniqueName: \"kubernetes.io/projected/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-kube-api-access-bpjd4\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.113834 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.113788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27009180-8547-4131-8824-d912c8fe24a3-config-volume\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214242 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214210 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqnzm\" (UniqueName: \"kubernetes.io/projected/27009180-8547-4131-8824-d912c8fe24a3-kube-api-access-zqnzm\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214242 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214246 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjd4\" (UniqueName: \"kubernetes.io/projected/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-kube-api-access-bpjd4\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.214457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27009180-8547-4131-8824-d912c8fe24a3-config-volume\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27009180-8547-4131-8824-d912c8fe24a3-tmp-dir\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214457 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:05.214392 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:05.214665 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:05.214467 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:05.714447995 +0000 UTC m=+35.560436866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:05.214665 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214400 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-cert\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.214913 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214895 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27009180-8547-4131-8824-d912c8fe24a3-tmp-dir\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.214983 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.214953 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27009180-8547-4131-8824-d912c8fe24a3-config-volume\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.219033 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.219011 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-cert\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.222628 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.222602 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqnzm\" (UniqueName: \"kubernetes.io/projected/27009180-8547-4131-8824-d912c8fe24a3-kube-api-access-zqnzm\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.222763 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.222690 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjd4\" (UniqueName: \"kubernetes.io/projected/480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8-kube-api-access-bpjd4\") pod \"ingress-canary-6wnzk\" (UID: \"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8\") " pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.383026 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.382961 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6wnzk" Jul 11 07:39:05.720091 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:05.720013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:05.720239 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:05.720175 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:05.720300 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:05.720241 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:06.720226057 +0000 UTC m=+36.566214929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:06.151871 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.151689 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6wnzk"] Jul 11 07:39:06.236850 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:39:06.236819 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480e6828_e7a8_4ff9_ab8a_8165bdf0a8a8.slice/crio-6b86a1e3b0f095911ab1a16c38a4fd172b76b18017df14d0821b9dcd2a49cf2d WatchSource:0}: Error finding container 6b86a1e3b0f095911ab1a16c38a4fd172b76b18017df14d0821b9dcd2a49cf2d: Status 404 returned error can't find the container with id 6b86a1e3b0f095911ab1a16c38a4fd172b76b18017df14d0821b9dcd2a49cf2d Jul 11 07:39:06.705523 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.705452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:06.705908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.705452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:06.705908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.705458 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:06.708716 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.708554 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 11 07:39:06.708716 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.708565 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs4vf\"" Jul 11 07:39:06.708716 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.708565 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 11 07:39:06.709714 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.709680 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92q58\"" Jul 11 07:39:06.709714 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.709722 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 11 07:39:06.709898 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.709824 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 11 07:39:06.726481 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.726465 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:06.726568 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:06.726557 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:06.726615 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:06.726603 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:08.726590999 +0000 UTC m=+38.572579870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:06.987774 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.987741 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="a63b3137947eab2ab73d33390cf6fa24ef3732ea112516b3da78ec2a5e606ccd" exitCode=0 Jul 11 07:39:06.987933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.987846 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"a63b3137947eab2ab73d33390cf6fa24ef3732ea112516b3da78ec2a5e606ccd"} Jul 11 07:39:06.989168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:06.989118 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6wnzk" event={"ID":"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8","Type":"ContainerStarted","Data":"6b86a1e3b0f095911ab1a16c38a4fd172b76b18017df14d0821b9dcd2a49cf2d"} Jul 11 07:39:07.994956 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:07.994919 2572 generic.go:356] "Generic (PLEG): container finished" podID="387f703e-cf0c-4e64-a468-cdb16936e32f" containerID="a78aaf983f50f53b6b2fdf4360043215f1d83d01097064bfefbc0e425325d196" exitCode=0 Jul 11 07:39:07.995410 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:07.994958 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerDied","Data":"a78aaf983f50f53b6b2fdf4360043215f1d83d01097064bfefbc0e425325d196"} Jul 11 07:39:08.740374 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:08.740346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:08.740503 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:08.740464 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:08.740548 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:08.740516 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:12.740497632 +0000 UTC m=+42.586486503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:09.001120 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:09.001038 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" event={"ID":"387f703e-cf0c-4e64-a468-cdb16936e32f","Type":"ContainerStarted","Data":"1951e59683bdce33cf76272f61deb99da97191472e9ecfbbfb8af3b27fe75497"} Jul 11 07:39:09.002638 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:09.002604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6wnzk" event={"ID":"480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8","Type":"ContainerStarted","Data":"318406368ec9ca9d8819bfc104f32d74f1aac5f56a1f87b82ddb1e297699a92f"} Jul 11 07:39:09.024015 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:09.023969 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4jv2k" podStartSLOduration=8.459169325 podStartE2EDuration="38.0239541s" totalImagesPullingTime="29.564784775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:09.022244023 +0000 UTC m=+38.868232930" watchObservedRunningTime="2026-07-11 07:39:09.0239541 +0000 UTC m=+38.869942993" Jul 11 07:39:09.037494 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:09.037450 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6wnzk" podStartSLOduration=1.792388696 podStartE2EDuration="4.037435966s" totalImagesPullingTime="2.24504727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:09.03695958 +0000 UTC m=+38.882948473" watchObservedRunningTime="2026-07-11 07:39:09.037435966 +0000 UTC m=+38.883424858" Jul 11 07:39:12.767727 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:12.767674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:12.768288 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:12.768069 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:12.768288 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:12.768149 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:20.768129114 +0000 UTC m=+50.614117987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:20.821675 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:20.821636 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:20.822166 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:20.821766 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:20.822166 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:20.821821 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:39:36.821807139 +0000 UTC m=+66.667796010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:30.975168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:30.975137 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xll5v" Jul 11 07:39:36.325335 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.324934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:36.325837 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.325436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:39:36.328444 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.328418 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 11 07:39:36.328571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.328457 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 11 07:39:36.335743 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:36.335725 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 11 07:39:36.335789 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:36.335775 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:40:40.335761183 +0000 UTC m=+130.181750057 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : secret "metrics-daemon-secret" not found Jul 11 07:39:36.338712 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.338689 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 11 07:39:36.351671 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.351648 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/8bb45ac4-39ba-4414-bdb0-286454433faf-kube-api-access-bk5hr\") pod \"network-check-target-8l8cg\" (UID: \"8bb45ac4-39ba-4414-bdb0-286454433faf\") " pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:36.425885 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.425840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:36.428456 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.428443 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 11 07:39:36.428828 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.428811 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92q58\"" Jul 11 07:39:36.436835 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.436819 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:36.438771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.438743 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21b00770-91a4-47b6-ae61-6eacb490ae55-original-pull-secret\") pod \"global-pull-secret-syncer-lhxbc\" (UID: \"21b00770-91a4-47b6-ae61-6eacb490ae55\") " pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:36.551725 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.551681 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l8cg"] Jul 11 07:39:36.554381 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:39:36.554356 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb45ac4_39ba_4414_bdb0_286454433faf.slice/crio-9cc26608fe20318dcc9ed2f80995d5498bbda6d42d31d124b15494e835609935 WatchSource:0}: Error finding container 9cc26608fe20318dcc9ed2f80995d5498bbda6d42d31d124b15494e835609935: Status 404 returned error can't find the container with id 9cc26608fe20318dcc9ed2f80995d5498bbda6d42d31d124b15494e835609935 Jul 11 07:39:36.722785 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.722727 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lhxbc" Jul 11 07:39:36.828174 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.828149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:39:36.828294 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:36.828261 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:39:36.828331 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:39:36.828323 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:40:08.828309318 +0000 UTC m=+98.674298189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:39:36.833770 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:36.833745 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lhxbc"] Jul 11 07:39:36.836648 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:39:36.836618 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b00770_91a4_47b6_ae61_6eacb490ae55.slice/crio-44669a2574bc1f0fb6fb7df5388ca6dd8b698c7405615523408a212f32a93483 WatchSource:0}: Error finding container 44669a2574bc1f0fb6fb7df5388ca6dd8b698c7405615523408a212f32a93483: Status 404 returned error can't find the container with id 44669a2574bc1f0fb6fb7df5388ca6dd8b698c7405615523408a212f32a93483 Jul 11 07:39:37.065208 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:37.065166 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lhxbc" event={"ID":"21b00770-91a4-47b6-ae61-6eacb490ae55","Type":"ContainerStarted","Data":"44669a2574bc1f0fb6fb7df5388ca6dd8b698c7405615523408a212f32a93483"} Jul 11 07:39:37.066105 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:37.066083 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l8cg" event={"ID":"8bb45ac4-39ba-4414-bdb0-286454433faf","Type":"ContainerStarted","Data":"9cc26608fe20318dcc9ed2f80995d5498bbda6d42d31d124b15494e835609935"} Jul 11 07:39:42.080465 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:42.080398 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l8cg" event={"ID":"8bb45ac4-39ba-4414-bdb0-286454433faf","Type":"ContainerStarted","Data":"067db424a817e96c0005cc244b50f61d04eaed2efea792b5b3c6084436af22d3"} Jul 11 07:39:42.080917 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:42.080523 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:39:42.081820 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:42.081795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lhxbc" event={"ID":"21b00770-91a4-47b6-ae61-6eacb490ae55","Type":"ContainerStarted","Data":"25cdc040b3d459eed96935f292177e2f9c8696eafcb240f0082f4966338d1982"} Jul 11 07:39:42.094562 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:42.094511 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8l8cg" podStartSLOduration=66.029302057 podStartE2EDuration="1m11.09449945s" totalImagesPullingTime="5.065197393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:42.094445483 +0000 UTC m=+71.940434390" watchObservedRunningTime="2026-07-11 07:39:42.09449945 +0000 UTC m=+71.940488342" Jul 11 07:39:42.107225 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:39:42.107185 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lhxbc" podStartSLOduration=66.312920795 podStartE2EDuration="1m11.1071739s" totalImagesPullingTime="4.794253105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:39:42.10672828 +0000 UTC m=+71.952717172" watchObservedRunningTime="2026-07-11 07:39:42.1071739 +0000 UTC m=+71.953162793" Jul 11 07:40:08.841654 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:40:08.841610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:40:08.842033 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:40:08.841778 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:40:08.842033 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:40:08.841854 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:41:12.841837399 +0000 UTC m=+162.687826275 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:40:13.086343 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:40:13.086310 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8l8cg" Jul 11 07:40:40.353213 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:40:40.353171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:40:40.353724 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:40:40.353322 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 11 07:40:40.353724 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:40:40.353386 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs podName:feffa75b-07b6-4079-9ed3-7f56aa5897cc nodeName:}" failed. No retries permitted until 2026-07-11 07:42:42.353370472 +0000 UTC m=+252.199359343 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs") pod "network-metrics-daemon-mcqm7" (UID: "feffa75b-07b6-4079-9ed3-7f56aa5897cc") : secret "metrics-daemon-secret" not found Jul 11 07:41:08.070197 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:41:08.070145 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qzl8f" podUID="27009180-8547-4131-8824-d912c8fe24a3" Jul 11 07:41:08.284662 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:41:08.284630 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:41:09.716242 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:41:09.716185 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mcqm7" podUID="feffa75b-07b6-4079-9ed3-7f56aa5897cc" Jul 11 07:41:12.872571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:41:12.872529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:41:12.872965 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:41:12.872681 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 11 07:41:12.872965 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:41:12.872768 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls podName:27009180-8547-4131-8824-d912c8fe24a3 nodeName:}" failed. No retries permitted until 2026-07-11 07:43:14.87275139 +0000 UTC m=+284.718740262 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls") pod "dns-default-qzl8f" (UID: "27009180-8547-4131-8824-d912c8fe24a3") : secret "dns-default-metrics-tls" not found Jul 11 07:41:20.706555 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:41:20.706523 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:42:12.350876 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.350843 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb"] Jul 11 07:42:12.352550 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.352533 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-r9jgt"] Jul 11 07:42:12.352687 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.352669 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.354307 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.354287 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.356317 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.356295 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 11 07:42:12.356423 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.356329 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 11 07:42:12.356477 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.356443 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zlnxk\"" Jul 11 07:42:12.357631 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357613 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 11 07:42:12.357758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357636 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:42:12.357758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357613 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 11 07:42:12.357758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357734 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 11 07:42:12.357929 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357868 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 11 07:42:12.357985 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.357947 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 11 07:42:12.358100 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.358084 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zknpv\"" Jul 11 07:42:12.362154 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.362138 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 11 07:42:12.371908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.371886 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb"] Jul 11 07:42:12.375463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.375441 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-r9jgt"] Jul 11 07:42:12.446029 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.445994 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m"] Jul 11 07:42:12.447716 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.447685 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m"] Jul 11 07:42:12.447838 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.447812 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" Jul 11 07:42:12.449631 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.449610 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-qqfcl"] Jul 11 07:42:12.449757 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.449733 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.450231 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.450213 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sbqth\"" Jul 11 07:42:12.451217 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.451200 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-ddfc4b4dd-5n58n"] Jul 11 07:42:12.451316 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.451301 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.452353 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452334 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-tmp\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/117717ba-3261-43a0-a194-5f464ce44c48-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.452443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452423 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dsdg\" (UniqueName: \"kubernetes.io/projected/117717ba-3261-43a0-a194-5f464ce44c48-kube-api-access-6dsdg\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452489 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjpn8\" (UniqueName: \"kubernetes.io/projected/358c222b-66c5-423c-9cd1-699f17e9e22c-kube-api-access-bjpn8\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452538 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-snapshots\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452539 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452549 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 11 07:42:12.452605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452599 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/358c222b-66c5-423c-9cd1-699f17e9e22c-serving-cert\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.452888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452637 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/117717ba-3261-43a0-a194-5f464ce44c48-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.452888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452833 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:42:12.452888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452883 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sjftz\"" Jul 11 07:42:12.452991 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.452916 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.453630 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.453613 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 11 07:42:12.453886 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.453865 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 11 07:42:12.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.453947 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7lwrm\"" Jul 11 07:42:12.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.453973 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 11 07:42:12.454003 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.453995 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:42:12.455616 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455596 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 11 07:42:12.455722 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455633 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 11 07:42:12.455722 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455648 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 11 07:42:12.455835 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455724 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 11 07:42:12.455976 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455959 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 11 07:42:12.456038 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.455983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 11 07:42:12.456777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.456761 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ptb9x\"" Jul 11 07:42:12.463050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.463020 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m"] Jul 11 07:42:12.463654 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.463630 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 11 07:42:12.463893 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.463877 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m"] Jul 11 07:42:12.466832 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.466811 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-qqfcl"] Jul 11 07:42:12.467908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.467886 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ddfc4b4dd-5n58n"] Jul 11 07:42:12.553831 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553798 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dsdg\" (UniqueName: \"kubernetes.io/projected/117717ba-3261-43a0-a194-5f464ce44c48-kube-api-access-6dsdg\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.553831 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjtmp\" (UniqueName: \"kubernetes.io/projected/c59d103b-558f-4aab-b3b5-828219f18c16-kube-api-access-vjtmp\") pod \"network-check-source-869dcfcdcd-r7f5m\" (UID: \"c59d103b-558f-4aab-b3b5-828219f18c16\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553876 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-snapshots\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553903 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-config\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553927 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.553962 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-default-certificate\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-serving-cert\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.554083 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554072 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knfh4\" (UniqueName: \"kubernetes.io/projected/a8597305-d702-404e-a359-55030ad011fa-kube-api-access-knfh4\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554093 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr6bg\" (UniqueName: \"kubernetes.io/projected/3001ea51-9ac9-4cb7-859d-751553e88bc5-kube-api-access-xr6bg\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjpn8\" (UniqueName: \"kubernetes.io/projected/358c222b-66c5-423c-9cd1-699f17e9e22c-kube-api-access-bjpn8\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554216 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4425x\" (UniqueName: \"kubernetes.io/projected/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-kube-api-access-4425x\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/358c222b-66c5-423c-9cd1-699f17e9e22c-serving-cert\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/117717ba-3261-43a0-a194-5f464ce44c48-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554332 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-stats-auth\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554376 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-trusted-ca\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554405 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554429 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554457 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-tmp\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554517 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/117717ba-3261-43a0-a194-5f464ce44c48-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.554573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554522 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-snapshots\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554964 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554619 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.554964 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554782 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/358c222b-66c5-423c-9cd1-699f17e9e22c-tmp\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.555034 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.554998 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/117717ba-3261-43a0-a194-5f464ce44c48-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.556053 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.556031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c222b-66c5-423c-9cd1-699f17e9e22c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.556462 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.556443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/358c222b-66c5-423c-9cd1-699f17e9e22c-serving-cert\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.556632 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.556607 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/117717ba-3261-43a0-a194-5f464ce44c48-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.563448 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.563430 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dsdg\" (UniqueName: \"kubernetes.io/projected/117717ba-3261-43a0-a194-5f464ce44c48-kube-api-access-6dsdg\") pod \"kube-storage-version-migrator-operator-799cb7f958-k4dkb\" (UID: \"117717ba-3261-43a0-a194-5f464ce44c48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.563569 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.563548 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjpn8\" (UniqueName: \"kubernetes.io/projected/358c222b-66c5-423c-9cd1-699f17e9e22c-kube-api-access-bjpn8\") pod \"insights-operator-68d9dd94cf-r9jgt\" (UID: \"358c222b-66c5-423c-9cd1-699f17e9e22c\") " pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.655213 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655132 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.655213 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.655213 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655191 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjtmp\" (UniqueName: \"kubernetes.io/projected/c59d103b-558f-4aab-b3b5-828219f18c16-kube-api-access-vjtmp\") pod \"network-check-source-869dcfcdcd-r7f5m\" (UID: \"c59d103b-558f-4aab-b3b5-828219f18c16\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655222 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-config\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:12.655281 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:12.655297 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-default-certificate\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:12.655342 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:13.15532805 +0000 UTC m=+223.001316920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : secret "router-metrics-certs-default" not found Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:12.655366 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls podName:3001ea51-9ac9-4cb7-859d-751553e88bc5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:13.15535669 +0000 UTC m=+223.001345561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls") pod "cluster-samples-operator-65db858894-l485m" (UID: "3001ea51-9ac9-4cb7-859d-751553e88bc5") : secret "samples-operator-tls" not found Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655418 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-serving-cert\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655439 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knfh4\" (UniqueName: \"kubernetes.io/projected/a8597305-d702-404e-a359-55030ad011fa-kube-api-access-knfh4\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.655463 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr6bg\" (UniqueName: \"kubernetes.io/projected/3001ea51-9ac9-4cb7-859d-751553e88bc5-kube-api-access-xr6bg\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.656044 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:12.655491 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:13.155471345 +0000 UTC m=+223.001460233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : configmap references non-existent config key: service-ca.crt Jul 11 07:42:12.656044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655540 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4425x\" (UniqueName: \"kubernetes.io/projected/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-kube-api-access-4425x\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.656044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-stats-auth\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.656044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.655610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-trusted-ca\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.656240 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.656043 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-config\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.656456 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.656435 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-trusted-ca\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.657584 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.657563 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-serving-cert\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.657832 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.657813 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-default-certificate\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.657877 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.657817 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-stats-auth\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.662219 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.662187 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" Jul 11 07:42:12.663725 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.663678 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr6bg\" (UniqueName: \"kubernetes.io/projected/3001ea51-9ac9-4cb7-859d-751553e88bc5-kube-api-access-xr6bg\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:12.664015 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.663984 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knfh4\" (UniqueName: \"kubernetes.io/projected/a8597305-d702-404e-a359-55030ad011fa-kube-api-access-knfh4\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:12.664509 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.664486 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4425x\" (UniqueName: \"kubernetes.io/projected/2329a2f5-b6c0-4b11-bfa4-ea93e65ada00-kube-api-access-4425x\") pod \"console-operator-7866497494-qqfcl\" (UID: \"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00\") " pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.664747 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.664730 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjtmp\" (UniqueName: \"kubernetes.io/projected/c59d103b-558f-4aab-b3b5-828219f18c16-kube-api-access-vjtmp\") pod \"network-check-source-869dcfcdcd-r7f5m\" (UID: \"c59d103b-558f-4aab-b3b5-828219f18c16\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" Jul 11 07:42:12.667354 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.667337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" Jul 11 07:42:12.759313 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.759283 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" Jul 11 07:42:12.778328 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.778294 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:12.793765 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.793571 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb"] Jul 11 07:42:12.797937 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:12.797907 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117717ba_3261_43a0_a194_5f464ce44c48.slice/crio-58e729a9229fdb329116ee36e261e3d193766f994ab2d4ba8622ae5ce0563a06 WatchSource:0}: Error finding container 58e729a9229fdb329116ee36e261e3d193766f994ab2d4ba8622ae5ce0563a06: Status 404 returned error can't find the container with id 58e729a9229fdb329116ee36e261e3d193766f994ab2d4ba8622ae5ce0563a06 Jul 11 07:42:12.814426 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.814356 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-r9jgt"] Jul 11 07:42:12.817570 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:12.817536 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c222b_66c5_423c_9cd1_699f17e9e22c.slice/crio-59c2aa78724645f2003a07050313c26a6ac007225bbed0c0e7798c1a3a5d04de WatchSource:0}: Error finding container 59c2aa78724645f2003a07050313c26a6ac007225bbed0c0e7798c1a3a5d04de: Status 404 returned error can't find the container with id 59c2aa78724645f2003a07050313c26a6ac007225bbed0c0e7798c1a3a5d04de Jul 11 07:42:12.891219 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.891187 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m"] Jul 11 07:42:12.894322 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:12.894293 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59d103b_558f_4aab_b3b5_828219f18c16.slice/crio-eaa4b8285a7cfa0ecda6cbcbb29f3ea010bcaa51cb8a3ee895bbf87e43a63e1e WatchSource:0}: Error finding container eaa4b8285a7cfa0ecda6cbcbb29f3ea010bcaa51cb8a3ee895bbf87e43a63e1e: Status 404 returned error can't find the container with id eaa4b8285a7cfa0ecda6cbcbb29f3ea010bcaa51cb8a3ee895bbf87e43a63e1e Jul 11 07:42:12.910555 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:12.910527 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-qqfcl"] Jul 11 07:42:12.921643 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:12.921618 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2329a2f5_b6c0_4b11_bfa4_ea93e65ada00.slice/crio-dcbef9c491482432bb3e89c745757754bcc3831aa6252c9531c4666823d8b414 WatchSource:0}: Error finding container dcbef9c491482432bb3e89c745757754bcc3831aa6252c9531c4666823d8b414: Status 404 returned error can't find the container with id dcbef9c491482432bb3e89c745757754bcc3831aa6252c9531c4666823d8b414 Jul 11 07:42:13.160174 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.160081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:13.160341 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.160224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:13.160341 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:13.160250 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:14.160232281 +0000 UTC m=+224.006221156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : configmap references non-existent config key: service-ca.crt Jul 11 07:42:13.160341 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.160296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:13.160341 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:13.160308 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 11 07:42:13.160607 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:13.160354 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:14.160339436 +0000 UTC m=+224.006328311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : secret "router-metrics-certs-default" not found Jul 11 07:42:13.160607 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:13.160398 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 11 07:42:13.160607 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:13.160434 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls podName:3001ea51-9ac9-4cb7-859d-751553e88bc5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:14.160422434 +0000 UTC m=+224.006411329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls") pod "cluster-samples-operator-65db858894-l485m" (UID: "3001ea51-9ac9-4cb7-859d-751553e88bc5") : secret "samples-operator-tls" not found Jul 11 07:42:13.442407 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.442149 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" event={"ID":"c59d103b-558f-4aab-b3b5-828219f18c16","Type":"ContainerStarted","Data":"3b5fb9e35e0f188de49766897b404879fda4fee9a67f3b9eb1d292500a250a55"} Jul 11 07:42:13.442407 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.442204 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" event={"ID":"c59d103b-558f-4aab-b3b5-828219f18c16","Type":"ContainerStarted","Data":"eaa4b8285a7cfa0ecda6cbcbb29f3ea010bcaa51cb8a3ee895bbf87e43a63e1e"} Jul 11 07:42:13.445144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.445062 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" event={"ID":"358c222b-66c5-423c-9cd1-699f17e9e22c","Type":"ContainerStarted","Data":"59c2aa78724645f2003a07050313c26a6ac007225bbed0c0e7798c1a3a5d04de"} Jul 11 07:42:13.447885 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.447859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qqfcl" event={"ID":"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00","Type":"ContainerStarted","Data":"dcbef9c491482432bb3e89c745757754bcc3831aa6252c9531c4666823d8b414"} Jul 11 07:42:13.450271 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.450246 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" event={"ID":"117717ba-3261-43a0-a194-5f464ce44c48","Type":"ContainerStarted","Data":"58e729a9229fdb329116ee36e261e3d193766f994ab2d4ba8622ae5ce0563a06"} Jul 11 07:42:13.457018 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:13.456964 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r7f5m" podStartSLOduration=1.4569442719999999 podStartE2EDuration="1.456944272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:13.456264027 +0000 UTC m=+223.302252921" watchObservedRunningTime="2026-07-11 07:42:13.456944272 +0000 UTC m=+223.302933165" Jul 11 07:42:14.169944 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:14.169903 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:14.170138 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:14.169988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:14.170138 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:14.170011 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:14.170138 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:14.170091 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:16.170069409 +0000 UTC m=+226.016058281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : configmap references non-existent config key: service-ca.crt Jul 11 07:42:14.170138 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:14.170119 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 11 07:42:14.170138 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:14.170126 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 11 07:42:14.170399 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:14.170168 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls podName:3001ea51-9ac9-4cb7-859d-751553e88bc5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:16.170154284 +0000 UTC m=+226.016143167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls") pod "cluster-samples-operator-65db858894-l485m" (UID: "3001ea51-9ac9-4cb7-859d-751553e88bc5") : secret "samples-operator-tls" not found Jul 11 07:42:14.170399 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:14.170181 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:16.170175074 +0000 UTC m=+226.016163949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : secret "router-metrics-certs-default" not found Jul 11 07:42:16.186532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.186490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:16.186532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.186530 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.186579 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:16.186641 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:16.186721 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.186693049 +0000 UTC m=+230.032681920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : configmap references non-existent config key: service-ca.crt Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:16.186740 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.186731899 +0000 UTC m=+230.032720770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : secret "router-metrics-certs-default" not found Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:16.186741 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 11 07:42:16.187030 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:16.186811 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls podName:3001ea51-9ac9-4cb7-859d-751553e88bc5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.186794287 +0000 UTC m=+230.032783162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls") pod "cluster-samples-operator-65db858894-l485m" (UID: "3001ea51-9ac9-4cb7-859d-751553e88bc5") : secret "samples-operator-tls" not found Jul 11 07:42:16.461034 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.460935 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" event={"ID":"358c222b-66c5-423c-9cd1-699f17e9e22c","Type":"ContainerStarted","Data":"2fa7004214b9be510ad7d4b300eddd24467e63263e0d47a501a7c7a381a374fa"} Jul 11 07:42:16.462496 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.462476 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/0.log" Jul 11 07:42:16.462613 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.462511 2572 generic.go:356] "Generic (PLEG): container finished" podID="2329a2f5-b6c0-4b11-bfa4-ea93e65ada00" containerID="5d7e40e137c4768a1ac14601ed57f4cfe23f17c7a702d7d49d0aca9c916fbc06" exitCode=255 Jul 11 07:42:16.462613 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.462583 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qqfcl" event={"ID":"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00","Type":"ContainerDied","Data":"5d7e40e137c4768a1ac14601ed57f4cfe23f17c7a702d7d49d0aca9c916fbc06"} Jul 11 07:42:16.462799 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.462787 2572 scope.go:117] "RemoveContainer" containerID="5d7e40e137c4768a1ac14601ed57f4cfe23f17c7a702d7d49d0aca9c916fbc06" Jul 11 07:42:16.464077 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.464048 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" event={"ID":"117717ba-3261-43a0-a194-5f464ce44c48","Type":"ContainerStarted","Data":"c731fd0aadd96cf4a0527f7b2866bdce7617c41caa21cbd99773fc99f0a29f7f"} Jul 11 07:42:16.477069 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:16.477027 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" podStartSLOduration=1.581973077 podStartE2EDuration="4.477013912s" totalImagesPullingTime="2.895040835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:16.476153147 +0000 UTC m=+226.322142040" watchObservedRunningTime="2026-07-11 07:42:16.477013912 +0000 UTC m=+226.323002783" Jul 11 07:42:17.220685 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.220634 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" podStartSLOduration=2.299407089 podStartE2EDuration="5.220609156s" totalImagesPullingTime="2.921202067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:16.506621309 +0000 UTC m=+226.352610204" watchObservedRunningTime="2026-07-11 07:42:17.220609156 +0000 UTC m=+227.066598067" Jul 11 07:42:17.221303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.221287 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-glsf8"] Jul 11 07:42:17.224437 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.224413 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.226806 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.226785 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mjxk7\"" Jul 11 07:42:17.226934 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.226903 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 11 07:42:17.227809 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.227791 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 11 07:42:17.235638 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.235612 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-glsf8"] Jul 11 07:42:17.296286 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.296247 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-data-volume\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.296286 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.296287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.296498 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.296368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h2nx\" (UniqueName: \"kubernetes.io/projected/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-api-access-2h2nx\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.296498 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.296416 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.296498 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.296435 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-crio-socket\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397035 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397000 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397035 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397034 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-crio-socket\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-data-volume\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397099 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.397114 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397138 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h2nx\" (UniqueName: \"kubernetes.io/projected/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-api-access-2h2nx\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.397177 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:17.897159344 +0000 UTC m=+227.743148218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : configmap "kube-rbac-proxy" not found Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397180 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-crio-socket\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.397246 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.397234 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 11 07:42:17.397466 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.397291 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:17.897273874 +0000 UTC m=+227.743262750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : secret "insights-runtime-extractor-tls" not found Jul 11 07:42:17.397509 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.397471 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-data-volume\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.406525 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.406501 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h2nx\" (UniqueName: \"kubernetes.io/projected/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-api-access-2h2nx\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.468585 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.468555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:42:17.469014 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.469000 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/0.log" Jul 11 07:42:17.469066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.469035 2572 generic.go:356] "Generic (PLEG): container finished" podID="2329a2f5-b6c0-4b11-bfa4-ea93e65ada00" containerID="d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3" exitCode=255 Jul 11 07:42:17.469126 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.469111 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qqfcl" event={"ID":"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00","Type":"ContainerDied","Data":"d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3"} Jul 11 07:42:17.469176 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.469152 2572 scope.go:117] "RemoveContainer" containerID="5d7e40e137c4768a1ac14601ed57f4cfe23f17c7a702d7d49d0aca9c916fbc06" Jul 11 07:42:17.469417 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.469393 2572 scope.go:117] "RemoveContainer" containerID="d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3" Jul 11 07:42:17.469607 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.469586 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qqfcl_openshift-console-operator(2329a2f5-b6c0-4b11-bfa4-ea93e65ada00)\"" pod="openshift-console-operator/console-operator-7866497494-qqfcl" podUID="2329a2f5-b6c0-4b11-bfa4-ea93e65ada00" Jul 11 07:42:17.902925 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.902877 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.903176 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:17.902955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:17.903176 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.903035 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 11 07:42:17.903176 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.903059 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 11 07:42:17.903176 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.903102 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:18.90308913 +0000 UTC m=+228.749078001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : secret "insights-runtime-extractor-tls" not found Jul 11 07:42:17.903176 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:17.903114 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:18.903107926 +0000 UTC m=+228.749096797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : configmap "kube-rbac-proxy" not found Jul 11 07:42:18.473541 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:18.473512 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:42:18.473929 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:18.473828 2572 scope.go:117] "RemoveContainer" containerID="d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3" Jul 11 07:42:18.473985 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:18.473966 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qqfcl_openshift-console-operator(2329a2f5-b6c0-4b11-bfa4-ea93e65ada00)\"" pod="openshift-console-operator/console-operator-7866497494-qqfcl" podUID="2329a2f5-b6c0-4b11-bfa4-ea93e65ada00" Jul 11 07:42:18.912282 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:18.912250 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:18.912457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:18.912310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:18.912457 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:18.912375 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 11 07:42:18.912457 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:18.912388 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 11 07:42:18.912457 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:18.912441 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.912427082 +0000 UTC m=+230.758415957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : secret "insights-runtime-extractor-tls" not found Jul 11 07:42:18.912457 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:18.912454 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:20.912448075 +0000 UTC m=+230.758436946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : configmap "kube-rbac-proxy" not found Jul 11 07:42:19.805085 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:19.805061 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h7j88_70fd8587-ba55-4187-a314-4fa49e388105/dns-node-resolver/0.log" Jul 11 07:42:20.225321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:20.225242 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:20.225321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:20.225273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:20.225321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:20.225322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:20.225522 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.225382 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 11 07:42:20.225522 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.225440 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:28.225424617 +0000 UTC m=+238.071413488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : secret "router-metrics-certs-default" not found Jul 11 07:42:20.225522 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.225445 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 11 07:42:20.225522 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.225454 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle podName:a8597305-d702-404e-a359-55030ad011fa nodeName:}" failed. No retries permitted until 2026-07-11 07:42:28.2254477 +0000 UTC m=+238.071436571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle") pod "router-default-ddfc4b4dd-5n58n" (UID: "a8597305-d702-404e-a359-55030ad011fa") : configmap references non-existent config key: service-ca.crt Jul 11 07:42:20.225522 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.225484 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls podName:3001ea51-9ac9-4cb7-859d-751553e88bc5 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:28.225471718 +0000 UTC m=+238.071460592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls") pod "cluster-samples-operator-65db858894-l485m" (UID: "3001ea51-9ac9-4cb7-859d-751553e88bc5") : secret "samples-operator-tls" not found Jul 11 07:42:20.930940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:20.930901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:20.931361 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:20.930998 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:20.931361 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.931054 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 11 07:42:20.931361 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.931100 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 11 07:42:20.931361 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.931113 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:24.931099014 +0000 UTC m=+234.777087888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : secret "insights-runtime-extractor-tls" not found Jul 11 07:42:20.931361 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:20.931151 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:24.931134614 +0000 UTC m=+234.777123486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : configmap "kube-rbac-proxy" not found Jul 11 07:42:21.204804 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:21.204731 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nk6bk_ded7f277-d188-4a8b-b232-80d7fabde531/node-ca/0.log" Jul 11 07:42:21.604556 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:21.604530 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6wnzk_480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8/serve-healthcheck-canary/0.log" Jul 11 07:42:22.607087 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:22.607045 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-k4dkb_117717ba-3261-43a0-a194-5f464ce44c48/kube-storage-version-migrator-operator/0.log" Jul 11 07:42:22.779391 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:22.779356 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:22.779564 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:22.779410 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:22.779774 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:22.779762 2572 scope.go:117] "RemoveContainer" containerID="d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3" Jul 11 07:42:22.779917 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:22.779899 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qqfcl_openshift-console-operator(2329a2f5-b6c0-4b11-bfa4-ea93e65ada00)\"" pod="openshift-console-operator/console-operator-7866497494-qqfcl" podUID="2329a2f5-b6c0-4b11-bfa4-ea93e65ada00" Jul 11 07:42:24.965044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:24.965011 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:24.965430 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:24.965069 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:24.965430 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:24.965123 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 11 07:42:24.965430 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:24.965169 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 11 07:42:24.965430 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:24.965198 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:32.965167653 +0000 UTC m=+242.811156527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : configmap "kube-rbac-proxy" not found Jul 11 07:42:24.965430 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:24.965215 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls podName:84ea31ae-0ba6-46f5-a661-7bb96290d8ee nodeName:}" failed. No retries permitted until 2026-07-11 07:42:32.965204913 +0000 UTC m=+242.811193784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glsf8" (UID: "84ea31ae-0ba6-46f5-a661-7bb96290d8ee") : secret "insights-runtime-extractor-tls" not found Jul 11 07:42:28.291840 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.291789 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:28.292332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.291902 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:28.292332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.291929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:28.292426 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.292384 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8597305-d702-404e-a359-55030ad011fa-service-ca-bundle\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:28.294225 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.294204 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3001ea51-9ac9-4cb7-859d-751553e88bc5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-l485m\" (UID: \"3001ea51-9ac9-4cb7-859d-751553e88bc5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:28.294314 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.294226 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8597305-d702-404e-a359-55030ad011fa-metrics-certs\") pod \"router-default-ddfc4b4dd-5n58n\" (UID: \"a8597305-d702-404e-a359-55030ad011fa\") " pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:28.372035 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.371992 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" Jul 11 07:42:28.382644 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.382618 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:28.511733 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.511687 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m"] Jul 11 07:42:28.527653 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:28.527626 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ddfc4b4dd-5n58n"] Jul 11 07:42:28.530382 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:28.530354 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8597305_d702_404e_a359_55030ad011fa.slice/crio-db8c125c14b017b55e044a39b00ca84e82be143d20b1ef31f8c3e24c6d4dac0c WatchSource:0}: Error finding container db8c125c14b017b55e044a39b00ca84e82be143d20b1ef31f8c3e24c6d4dac0c: Status 404 returned error can't find the container with id db8c125c14b017b55e044a39b00ca84e82be143d20b1ef31f8c3e24c6d4dac0c Jul 11 07:42:29.510989 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:29.510952 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" event={"ID":"a8597305-d702-404e-a359-55030ad011fa","Type":"ContainerStarted","Data":"a5efbd96fd1b461d0665c675ede126ea481ef380bdb36aa022b09f1ab93effed"} Jul 11 07:42:29.511455 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:29.511004 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" event={"ID":"a8597305-d702-404e-a359-55030ad011fa","Type":"ContainerStarted","Data":"db8c125c14b017b55e044a39b00ca84e82be143d20b1ef31f8c3e24c6d4dac0c"} Jul 11 07:42:29.512269 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:29.512243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" event={"ID":"3001ea51-9ac9-4cb7-859d-751553e88bc5","Type":"ContainerStarted","Data":"34c40cfebee4ef933317055a97c9f98edff69d40f79c492ed088c806e4b56c54"} Jul 11 07:42:29.531354 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:29.531309 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" podStartSLOduration=17.531294418999998 podStartE2EDuration="17.531294419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:29.530168374 +0000 UTC m=+239.376157298" watchObservedRunningTime="2026-07-11 07:42:29.531294419 +0000 UTC m=+239.377283303" Jul 11 07:42:30.383725 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:30.383680 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:30.386235 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:30.386212 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:30.518381 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:30.518346 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" event={"ID":"3001ea51-9ac9-4cb7-859d-751553e88bc5","Type":"ContainerStarted","Data":"e5bf4f8b5598afa68e6054dcafd14413d5ee47b28e1dd5f3a514458ac0e25f93"} Jul 11 07:42:30.518755 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:30.518606 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:30.519865 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:30.519842 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-ddfc4b4dd-5n58n" Jul 11 07:42:31.522476 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:31.522439 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" event={"ID":"3001ea51-9ac9-4cb7-859d-751553e88bc5","Type":"ContainerStarted","Data":"cc1c648d00d10c17c306617ff36b57897515fe939bfbed7aef721ed928630a97"} Jul 11 07:42:31.539020 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:31.538977 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-l485m" podStartSLOduration=17.674484426 podStartE2EDuration="19.538964179s" totalImagesPullingTime="1.864479753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:31.537673705 +0000 UTC m=+241.383662600" watchObservedRunningTime="2026-07-11 07:42:31.538964179 +0000 UTC m=+241.384953121" Jul 11 07:42:33.028666 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.028617 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:33.029057 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.028776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:33.029162 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.029141 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:33.030925 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.030909 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84ea31ae-0ba6-46f5-a661-7bb96290d8ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glsf8\" (UID: \"84ea31ae-0ba6-46f5-a661-7bb96290d8ee\") " pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:33.136525 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.136496 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mjxk7\"" Jul 11 07:42:33.144717 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.144680 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-glsf8" Jul 11 07:42:33.270766 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.270735 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-glsf8"] Jul 11 07:42:33.273544 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:33.273516 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ea31ae_0ba6_46f5_a661_7bb96290d8ee.slice/crio-cfe128e56ca48b3c301af37208b4cee7c7e12775d785b637fc4ed50c4a2fb542 WatchSource:0}: Error finding container cfe128e56ca48b3c301af37208b4cee7c7e12775d785b637fc4ed50c4a2fb542: Status 404 returned error can't find the container with id cfe128e56ca48b3c301af37208b4cee7c7e12775d785b637fc4ed50c4a2fb542 Jul 11 07:42:33.529803 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:33.529725 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glsf8" event={"ID":"84ea31ae-0ba6-46f5-a661-7bb96290d8ee","Type":"ContainerStarted","Data":"cfe128e56ca48b3c301af37208b4cee7c7e12775d785b637fc4ed50c4a2fb542"} Jul 11 07:42:35.705840 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:35.705815 2572 scope.go:117] "RemoveContainer" containerID="d4da0e1aa545fa1af2770e62f39681748fe6493e253239e71b10e166a25d00a3" Jul 11 07:42:37.545531 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.545505 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:42:37.546004 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.545600 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qqfcl" event={"ID":"2329a2f5-b6c0-4b11-bfa4-ea93e65ada00","Type":"ContainerStarted","Data":"c45a7dd9526e7b292021d813e25d7768b96aebc5dfd339d61c320dcd07e1d709"} Jul 11 07:42:37.546004 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.545954 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:37.547388 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.547361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glsf8" event={"ID":"84ea31ae-0ba6-46f5-a661-7bb96290d8ee","Type":"ContainerStarted","Data":"62c92fb5cf36182d0bbfcad3793b0f2075a4e6c8b09681e0af6f0c63ad68d5f3"} Jul 11 07:42:37.550910 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.550895 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-qqfcl" Jul 11 07:42:37.561505 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:37.561464 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-qqfcl" podStartSLOduration=22.76258497 podStartE2EDuration="25.561449705s" totalImagesPullingTime="2.798864735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:37.560416982 +0000 UTC m=+247.406405874" watchObservedRunningTime="2026-07-11 07:42:37.561449705 +0000 UTC m=+247.407438598" Jul 11 07:42:38.553002 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.552961 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glsf8" event={"ID":"84ea31ae-0ba6-46f5-a661-7bb96290d8ee","Type":"ContainerStarted","Data":"e1e193dd15c9757fced37c7363949936eab2119e5d35c4c41bfd5e733e3629c9"} Jul 11 07:42:38.979829 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.979131 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-856c5d9b5c-hgdfv"] Jul 11 07:42:38.982831 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.982810 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:38.986385 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.986337 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 11 07:42:38.986657 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.986634 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4gzdp\"" Jul 11 07:42:38.986811 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.986768 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 11 07:42:38.986865 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.986849 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 11 07:42:38.992224 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.992199 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 11 07:42:38.999258 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:38.999234 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-856c5d9b5c-hgdfv"] Jul 11 07:42:39.082510 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082476 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-image-registry-private-configuration\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-tls\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-installation-pull-secrets\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082588 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcngw\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-kube-api-access-rcngw\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-trusted-ca\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-bound-sa-token\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082942 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082749 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-certificates\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.082942 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.082795 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f96658f9-74c3-42a4-aa52-6667cfe59c26-ca-trust-extracted\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183476 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183445 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-tls\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183476 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183485 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-installation-pull-secrets\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcngw\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-kube-api-access-rcngw\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183551 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-trusted-ca\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-bound-sa-token\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183898 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183823 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-certificates\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183898 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183877 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f96658f9-74c3-42a4-aa52-6667cfe59c26-ca-trust-extracted\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.183984 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.183971 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-image-registry-private-configuration\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.184322 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.184300 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f96658f9-74c3-42a4-aa52-6667cfe59c26-ca-trust-extracted\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.184671 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.184647 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-certificates\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.185829 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.185044 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f96658f9-74c3-42a4-aa52-6667cfe59c26-trusted-ca\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.189843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.186976 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-registry-tls\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.190635 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.190611 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-image-registry-private-configuration\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.190764 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.190734 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f96658f9-74c3-42a4-aa52-6667cfe59c26-installation-pull-secrets\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.193339 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.193314 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-bound-sa-token\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.193697 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.193674 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcngw\" (UniqueName: \"kubernetes.io/projected/f96658f9-74c3-42a4-aa52-6667cfe59c26-kube-api-access-rcngw\") pod \"image-registry-856c5d9b5c-hgdfv\" (UID: \"f96658f9-74c3-42a4-aa52-6667cfe59c26\") " pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:39.294326 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:39.294286 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:40.115696 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.115674 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-856c5d9b5c-hgdfv"] Jul 11 07:42:40.119622 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:40.119586 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96658f9_74c3_42a4_aa52_6667cfe59c26.slice/crio-ac7d589d86598d26655a1330e7afcaa5abc265f4c4a20fb46b57d7b030cec184 WatchSource:0}: Error finding container ac7d589d86598d26655a1330e7afcaa5abc265f4c4a20fb46b57d7b030cec184: Status 404 returned error can't find the container with id ac7d589d86598d26655a1330e7afcaa5abc265f4c4a20fb46b57d7b030cec184 Jul 11 07:42:40.563140 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.563100 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glsf8" event={"ID":"84ea31ae-0ba6-46f5-a661-7bb96290d8ee","Type":"ContainerStarted","Data":"1929d14e3bc11c77cb52c4013601261a2a01ded4767dd5bc40513d9264d8917f"} Jul 11 07:42:40.564542 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.564517 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" event={"ID":"f96658f9-74c3-42a4-aa52-6667cfe59c26","Type":"ContainerStarted","Data":"44b60406c7dbef2bca7e74af89bb5f2407115ade73ee015cb4c70e2b0d0f5e05"} Jul 11 07:42:40.564672 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.564559 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" event={"ID":"f96658f9-74c3-42a4-aa52-6667cfe59c26","Type":"ContainerStarted","Data":"ac7d589d86598d26655a1330e7afcaa5abc265f4c4a20fb46b57d7b030cec184"} Jul 11 07:42:40.564672 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.564644 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:42:40.583762 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.583680 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-glsf8" podStartSLOduration=16.944400555 podStartE2EDuration="23.583655683s" totalImagesPullingTime="6.639255128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 07:42:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:40.582313815 +0000 UTC m=+250.428302746" watchObservedRunningTime="2026-07-11 07:42:40.583655683 +0000 UTC m=+250.429644576" Jul 11 07:42:40.600022 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:40.599972 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" podStartSLOduration=2.599956722 podStartE2EDuration="2.599956722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:40.598567092 +0000 UTC m=+250.444555984" watchObservedRunningTime="2026-07-11 07:42:40.599956722 +0000 UTC m=+250.445945615" Jul 11 07:42:42.411182 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:42.411135 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:42:42.413389 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:42.413366 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feffa75b-07b6-4079-9ed3-7f56aa5897cc-metrics-certs\") pod \"network-metrics-daemon-mcqm7\" (UID: \"feffa75b-07b6-4079-9ed3-7f56aa5897cc\") " pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:42:42.609639 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:42.609608 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs4vf\"" Jul 11 07:42:42.617881 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:42.617863 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcqm7" Jul 11 07:42:42.759136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:42.759106 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mcqm7"] Jul 11 07:42:42.762206 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:42.762182 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeffa75b_07b6_4079_9ed3_7f56aa5897cc.slice/crio-89192a5d8306658e577b9562266b623de0337d73409a71524fecc23047461dc4 WatchSource:0}: Error finding container 89192a5d8306658e577b9562266b623de0337d73409a71524fecc23047461dc4: Status 404 returned error can't find the container with id 89192a5d8306658e577b9562266b623de0337d73409a71524fecc23047461dc4 Jul 11 07:42:43.332754 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.332721 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:42:43.335723 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.335682 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.338486 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.338459 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 11 07:42:43.338592 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.338537 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 11 07:42:43.338592 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.338555 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 11 07:42:43.339581 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.339565 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 11 07:42:43.339665 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.339595 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 11 07:42:43.339967 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.339952 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gqvhv\"" Jul 11 07:42:43.340059 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.339979 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 11 07:42:43.340059 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.340034 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 11 07:42:43.346149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.346129 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:42:43.419710 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419665 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.420149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419719 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksnv6\" (UniqueName: \"kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.420149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419781 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.420149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.420149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419873 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.420149 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.419895 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521110 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521072 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521177 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521198 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521478 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksnv6\" (UniqueName: \"kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.521963 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521937 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.522066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521973 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.522066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.521944 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.524208 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.524187 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.524318 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.524241 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.530514 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.530488 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksnv6\" (UniqueName: \"kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6\") pod \"console-66548b89-wsnjw\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.576637 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.576585 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcqm7" event={"ID":"feffa75b-07b6-4079-9ed3-7f56aa5897cc","Type":"ContainerStarted","Data":"89192a5d8306658e577b9562266b623de0337d73409a71524fecc23047461dc4"} Jul 11 07:42:43.645086 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.645001 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:43.783208 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:43.783177 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:42:43.785878 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:43.785847 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9246ff50_06f2_47ef_b3f7_93e9bf21bb2e.slice/crio-afd662618b6c49a631f09288da4fd869f7dad3015ab75aebf146fd5a42ce747c WatchSource:0}: Error finding container afd662618b6c49a631f09288da4fd869f7dad3015ab75aebf146fd5a42ce747c: Status 404 returned error can't find the container with id afd662618b6c49a631f09288da4fd869f7dad3015ab75aebf146fd5a42ce747c Jul 11 07:42:44.582308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:44.582268 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcqm7" event={"ID":"feffa75b-07b6-4079-9ed3-7f56aa5897cc","Type":"ContainerStarted","Data":"351de5c49db1907bd1aa142a080006a2b860bb96801cf9294b1ac6a766a6d667"} Jul 11 07:42:44.582759 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:44.582331 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcqm7" event={"ID":"feffa75b-07b6-4079-9ed3-7f56aa5897cc","Type":"ContainerStarted","Data":"3bfb17cb93b23939cc4bc667152fda9a732c8be0640e944ef6fe927b2150c850"} Jul 11 07:42:44.583416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:44.583378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66548b89-wsnjw" event={"ID":"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e","Type":"ContainerStarted","Data":"afd662618b6c49a631f09288da4fd869f7dad3015ab75aebf146fd5a42ce747c"} Jul 11 07:42:44.599142 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:44.599095 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mcqm7" podStartSLOduration=252.66092972 podStartE2EDuration="4m13.599081348s" totalImagesPullingTime="938.151628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:44.598254114 +0000 UTC m=+254.444243010" watchObservedRunningTime="2026-07-11 07:42:44.599081348 +0000 UTC m=+254.445070241" Jul 11 07:42:47.596183 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:47.596151 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66548b89-wsnjw" event={"ID":"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e","Type":"ContainerStarted","Data":"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c"} Jul 11 07:42:47.612569 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:47.612519 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66548b89-wsnjw" podStartSLOduration=1.2940503620000001 podStartE2EDuration="4.612503725s" totalImagesPullingTime="3.318453363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:47.612009859 +0000 UTC m=+257.457998752" watchObservedRunningTime="2026-07-11 07:42:47.612503725 +0000 UTC m=+257.458492618" Jul 11 07:42:48.066422 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.066385 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l"] Jul 11 07:42:48.069777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.069758 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.072462 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.072438 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 11 07:42:48.072611 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.072592 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 11 07:42:48.072720 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.072677 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 11 07:42:48.073742 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.073722 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 11 07:42:48.073838 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.073726 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-48mjb\"" Jul 11 07:42:48.073907 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.073838 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 11 07:42:48.078881 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.078862 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l"] Jul 11 07:42:48.086871 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.086848 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mc4dw"] Jul 11 07:42:48.089890 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.089871 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zllhq"] Jul 11 07:42:48.089999 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.089983 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.092942 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.092923 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 11 07:42:48.093042 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.093020 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 11 07:42:48.093042 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.093028 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.093123 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.093056 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8hmr4\"" Jul 11 07:42:48.093123 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.093075 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 11 07:42:48.095605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.095587 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-876tw\"" Jul 11 07:42:48.095835 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.095817 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 11 07:42:48.095909 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.095824 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 11 07:42:48.096070 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.096055 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 11 07:42:48.100497 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.100473 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mc4dw"] Jul 11 07:42:48.159212 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.159212 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjgml\" (UniqueName: \"kubernetes.io/projected/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-kube-api-access-rjgml\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159235 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159260 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-root\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159365 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-metrics-client-ca\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159393 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-sys\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7q8m\" (UniqueName: \"kubernetes.io/projected/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-api-access-p7q8m\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-accelerators-collector-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159468 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-wtmp\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159536 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9f1206a3-dacc-4ead-a8ad-9db92a016941-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159555 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsvv9\" (UniqueName: \"kubernetes.io/projected/25393902-e94b-4db6-9f88-0ccf0a2fb728-kube-api-access-dsvv9\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159608 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.159858 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159858 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159662 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-textfile\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.159858 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.159858 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.159741 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-accelerators-collector-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260303 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-wtmp\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9f1206a3-dacc-4ead-a8ad-9db92a016941-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsvv9\" (UniqueName: \"kubernetes.io/projected/25393902-e94b-4db6-9f88-0ccf0a2fb728-kube-api-access-dsvv9\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260434 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260473 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-wtmp\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-textfile\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.260526 2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260566 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.260591 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls podName:25393902-e94b-4db6-9f88-0ccf0a2fb728 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:48.760571576 +0000 UTC m=+258.606560453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls") pod "node-exporter-zllhq" (UID: "25393902-e94b-4db6-9f88-0ccf0a2fb728") : secret "node-exporter-tls" not found Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260664 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260696 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjgml\" (UniqueName: \"kubernetes.io/projected/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-kube-api-access-rjgml\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-root\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260822 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9f1206a3-dacc-4ead-a8ad-9db92a016941-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.260843 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260827 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260887 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-metrics-client-ca\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260901 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-textfile\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-sys\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260955 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-sys\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260965 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7q8m\" (UniqueName: \"kubernetes.io/projected/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-api-access-p7q8m\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.260991 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-accelerators-collector-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.261046 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25393902-e94b-4db6-9f88-0ccf0a2fb728-root\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.261118 2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.261173 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls podName:ebc58e6b-ce0e-429b-a7e9-1aef707b01fc nodeName:}" failed. No retries permitted until 2026-07-11 07:42:48.761158856 +0000 UTC m=+258.607147740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-lzn6l" (UID: "ebc58e6b-ce0e-429b-a7e9-1aef707b01fc") : secret "openshift-state-metrics-tls" not found Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.261119 2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.261204 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls podName:9f1206a3-dacc-4ead-a8ad-9db92a016941 nodeName:}" failed. No retries permitted until 2026-07-11 07:42:48.761195902 +0000 UTC m=+258.607184774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-mc4dw" (UID: "9f1206a3-dacc-4ead-a8ad-9db92a016941") : secret "kube-state-metrics-tls" not found Jul 11 07:42:48.261363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.261311 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.261912 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.261459 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25393902-e94b-4db6-9f88-0ccf0a2fb728-metrics-client-ca\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.261912 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.261462 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.261912 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.261777 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1206a3-dacc-4ead-a8ad-9db92a016941-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.263033 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.263296 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.263274 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.263422 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.263407 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.267909 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.267884 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsvv9\" (UniqueName: \"kubernetes.io/projected/25393902-e94b-4db6-9f88-0ccf0a2fb728-kube-api-access-dsvv9\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.269255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.269237 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjgml\" (UniqueName: \"kubernetes.io/projected/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-kube-api-access-rjgml\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.269339 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.269304 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7q8m\" (UniqueName: \"kubernetes.io/projected/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-api-access-p7q8m\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.765216 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.765181 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:48.765216 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.765221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:48.765635 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.765291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.765635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.765343 2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 11 07:42:48.765635 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:42:48.765408 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls podName:ebc58e6b-ce0e-429b-a7e9-1aef707b01fc nodeName:}" failed. No retries permitted until 2026-07-11 07:42:49.765388539 +0000 UTC m=+259.611377410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-lzn6l" (UID: "ebc58e6b-ce0e-429b-a7e9-1aef707b01fc") : secret "openshift-state-metrics-tls" not found Jul 11 07:42:48.767518 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.767496 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25393902-e94b-4db6-9f88-0ccf0a2fb728-node-exporter-tls\") pod \"node-exporter-zllhq\" (UID: \"25393902-e94b-4db6-9f88-0ccf0a2fb728\") " pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:48.767635 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:48.767618 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1206a3-dacc-4ead-a8ad-9db92a016941-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mc4dw\" (UID: \"9f1206a3-dacc-4ead-a8ad-9db92a016941\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:49.001325 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.001289 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" Jul 11 07:42:49.006943 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.006920 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zllhq" Jul 11 07:42:49.018294 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:49.018228 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25393902_e94b_4db6_9f88_0ccf0a2fb728.slice/crio-e17e9514ea98673afe5edfe9049db0df7967ceb359e6fe373940aceb46f63e1d WatchSource:0}: Error finding container e17e9514ea98673afe5edfe9049db0df7967ceb359e6fe373940aceb46f63e1d: Status 404 returned error can't find the container with id e17e9514ea98673afe5edfe9049db0df7967ceb359e6fe373940aceb46f63e1d Jul 11 07:42:49.137148 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.137122 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mc4dw"] Jul 11 07:42:49.139659 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:49.139630 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1206a3_dacc_4ead_a8ad_9db92a016941.slice/crio-e58312ea0ad9591f44d99ec89b7d320dd5ca270d4b4fcd377d840d38589a4933 WatchSource:0}: Error finding container e58312ea0ad9591f44d99ec89b7d320dd5ca270d4b4fcd377d840d38589a4933: Status 404 returned error can't find the container with id e58312ea0ad9591f44d99ec89b7d320dd5ca270d4b4fcd377d840d38589a4933 Jul 11 07:42:49.605369 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.605312 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zllhq" event={"ID":"25393902-e94b-4db6-9f88-0ccf0a2fb728","Type":"ContainerStarted","Data":"e17e9514ea98673afe5edfe9049db0df7967ceb359e6fe373940aceb46f63e1d"} Jul 11 07:42:49.607232 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.607199 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" event={"ID":"9f1206a3-dacc-4ead-a8ad-9db92a016941","Type":"ContainerStarted","Data":"e58312ea0ad9591f44d99ec89b7d320dd5ca270d4b4fcd377d840d38589a4933"} Jul 11 07:42:49.775315 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.775273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:49.777945 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.777919 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebc58e6b-ce0e-429b-a7e9-1aef707b01fc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lzn6l\" (UID: \"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:49.879622 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:49.879592 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" Jul 11 07:42:50.308172 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.308077 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l"] Jul 11 07:42:50.311350 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:50.311327 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc58e6b_ce0e_429b_a7e9_1aef707b01fc.slice/crio-ef1a7076e5938cd457757fa31d6d2b48cac54885a3b3739cfa39ef3303dbc8e4 WatchSource:0}: Error finding container ef1a7076e5938cd457757fa31d6d2b48cac54885a3b3739cfa39ef3303dbc8e4: Status 404 returned error can't find the container with id ef1a7076e5938cd457757fa31d6d2b48cac54885a3b3739cfa39ef3303dbc8e4 Jul 11 07:42:50.612483 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.612443 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" event={"ID":"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc","Type":"ContainerStarted","Data":"ee3b3ad1b30fa8a53b4f162a384b6f2e2d537a66b2f5e7a3dae9be1eb6a3c5a3"} Jul 11 07:42:50.612677 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.612521 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" event={"ID":"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc","Type":"ContainerStarted","Data":"a7e2df5cb6c46402578fc4632b430d303b1bbd4d8e12b08926627c9278be4a0e"} Jul 11 07:42:50.612677 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.612554 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" event={"ID":"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc","Type":"ContainerStarted","Data":"ef1a7076e5938cd457757fa31d6d2b48cac54885a3b3739cfa39ef3303dbc8e4"} Jul 11 07:42:50.614146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.614116 2572 generic.go:356] "Generic (PLEG): container finished" podID="25393902-e94b-4db6-9f88-0ccf0a2fb728" containerID="3e09f01132d7111d4bd30941785bc2316125a37eee0b0f5148146689b55dff91" exitCode=0 Jul 11 07:42:50.614273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.614190 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zllhq" event={"ID":"25393902-e94b-4db6-9f88-0ccf0a2fb728","Type":"ContainerDied","Data":"3e09f01132d7111d4bd30941785bc2316125a37eee0b0f5148146689b55dff91"} Jul 11 07:42:50.616392 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.616368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" event={"ID":"9f1206a3-dacc-4ead-a8ad-9db92a016941","Type":"ContainerStarted","Data":"3514c853db0723238662ab8a11129741d2cbdbfb0f876ab3f994f85e2ff7a852"} Jul 11 07:42:50.616509 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.616428 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" event={"ID":"9f1206a3-dacc-4ead-a8ad-9db92a016941","Type":"ContainerStarted","Data":"478ec6e025af838d03ac6da0f5ea31f945af4e67d162bbdc13c90eb875d12da9"} Jul 11 07:42:50.616509 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.616458 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" event={"ID":"9f1206a3-dacc-4ead-a8ad-9db92a016941","Type":"ContainerStarted","Data":"6d268f7013c66d1a1d68494f11386d4f4f35e8aef3b6e7c4ef31406757433fcc"} Jul 11 07:42:50.648824 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:50.648775 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-mc4dw" podStartSLOduration=1.567092815 podStartE2EDuration="2.648759739s" totalImagesPullingTime="1.081666924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:50.647038265 +0000 UTC m=+260.493027157" watchObservedRunningTime="2026-07-11 07:42:50.648759739 +0000 UTC m=+260.494748631" Jul 11 07:42:51.623444 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:51.623390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zllhq" event={"ID":"25393902-e94b-4db6-9f88-0ccf0a2fb728","Type":"ContainerStarted","Data":"2f3c35fb4fa84380bdd3f07fde38211d5d2bf10ed5e153e1f99f2cb44829350a"} Jul 11 07:42:51.623910 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:51.623461 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zllhq" event={"ID":"25393902-e94b-4db6-9f88-0ccf0a2fb728","Type":"ContainerStarted","Data":"21c0d5e7669fadad681454dfc22db170c9618b893fdef0fb37757fb1fb076cb3"} Jul 11 07:42:51.625442 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:51.625418 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" event={"ID":"ebc58e6b-ce0e-429b-a7e9-1aef707b01fc","Type":"ContainerStarted","Data":"b495e179cf5d7ce6117ec05802799bbb6b27cdb0ce0b65b129bdd8e23443acc1"} Jul 11 07:42:51.642354 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:51.642304 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zllhq" podStartSLOduration=2.781022316 podStartE2EDuration="3.642287643s" totalImagesPullingTime="789.184731ms" totalInitContainerRuntime="72.080596ms" isStatefulPod=true podCreationTimestamp="2026-07-11 07:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:51.640535303 +0000 UTC m=+261.486524208" watchObservedRunningTime="2026-07-11 07:42:51.642287643 +0000 UTC m=+261.488276535" Jul 11 07:42:51.658166 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:51.658115 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lzn6l" podStartSLOduration=2.640525257 podStartE2EDuration="3.658098s" totalImagesPullingTime="1.017572743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:51.656790793 +0000 UTC m=+261.502779708" watchObservedRunningTime="2026-07-11 07:42:51.658098 +0000 UTC m=+261.504086891" Jul 11 07:42:52.384204 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.383220 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-576994c976-gwgjv"] Jul 11 07:42:52.387624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.387573 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.390715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.390680 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p58c9\"" Jul 11 07:42:52.390939 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.390923 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 11 07:42:52.391025 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.391002 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 11 07:42:52.391848 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.391829 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 11 07:42:52.391943 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.391926 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 11 07:42:52.392226 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.392211 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cokfjuej5s2og\"" Jul 11 07:42:52.396185 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.396166 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-576994c976-gwgjv"] Jul 11 07:42:52.503014 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.502979 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-audit-log\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503045 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503079 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-metrics-server-audit-profiles\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503175 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-tls\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503280 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503206 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-client-certs\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503280 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-client-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.503280 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.503241 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ppnr\" (UniqueName: \"kubernetes.io/projected/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-kube-api-access-4ppnr\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603722 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-tls\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-client-certs\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-client-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603779 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ppnr\" (UniqueName: \"kubernetes.io/projected/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-kube-api-access-4ppnr\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603800 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-audit-log\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.603878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603847 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.604139 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.603879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-metrics-server-audit-profiles\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.604319 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.604283 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-audit-log\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.604627 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.604604 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.604814 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.604798 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-metrics-server-audit-profiles\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.606290 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.606259 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-client-certs\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.606623 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.606607 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-client-ca-bundle\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.606670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.606636 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-secret-metrics-server-tls\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.611647 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.611626 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ppnr\" (UniqueName: \"kubernetes.io/projected/e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05-kube-api-access-4ppnr\") pod \"metrics-server-576994c976-gwgjv\" (UID: \"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05\") " pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.697958 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.697875 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:42:52.839254 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:52.839225 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-576994c976-gwgjv"] Jul 11 07:42:52.842484 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:52.842457 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5a4a9_f884_4c0e_a0d8_62bbbb383b05.slice/crio-c1c92b40b84844134674aadb4171371bb5f9e06b0a23c557be19df6afdffb50b WatchSource:0}: Error finding container c1c92b40b84844134674aadb4171371bb5f9e06b0a23c557be19df6afdffb50b: Status 404 returned error can't find the container with id c1c92b40b84844134674aadb4171371bb5f9e06b0a23c557be19df6afdffb50b Jul 11 07:42:53.262282 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.262245 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg"] Jul 11 07:42:53.266620 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.266599 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.269630 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269598 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 11 07:42:53.269771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269661 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 11 07:42:53.269771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269671 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 11 07:42:53.269771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269604 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 11 07:42:53.269771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269605 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bbngg\"" Jul 11 07:42:53.270005 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.269991 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 11 07:42:53.275593 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.275571 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 11 07:42:53.277168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.277148 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg"] Jul 11 07:42:53.310727 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310659 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-serving-certs-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.310727 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlp96\" (UniqueName: \"kubernetes.io/projected/33cbdf41-5531-448d-a358-636f510b7950-kube-api-access-vlp96\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.310915 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310814 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-metrics-client-ca\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.310915 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-telemeter-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.311023 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310928 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.311023 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-federate-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.311023 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.310971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.311023 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.311017 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.411961 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.411918 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-telemeter-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412052 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-federate-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412079 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412146 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412109 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412350 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-serving-certs-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412350 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlp96\" (UniqueName: \"kubernetes.io/projected/33cbdf41-5531-448d-a358-636f510b7950-kube-api-access-vlp96\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.412350 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.412224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-metrics-client-ca\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.413292 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.413218 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.413754 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.413685 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-metrics-client-ca\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.414575 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.414551 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33cbdf41-5531-448d-a358-636f510b7950-serving-certs-ca-bundle\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.415273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.415244 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.415901 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.415875 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-federate-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.416080 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.416057 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.416312 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.416293 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/33cbdf41-5531-448d-a358-636f510b7950-telemeter-client-tls\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.421047 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.421004 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlp96\" (UniqueName: \"kubernetes.io/projected/33cbdf41-5531-448d-a358-636f510b7950-kube-api-access-vlp96\") pod \"telemeter-client-6cfd4dfc88-wz4dg\" (UID: \"33cbdf41-5531-448d-a358-636f510b7950\") " pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.578029 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.577949 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" Jul 11 07:42:53.638636 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.638602 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" event={"ID":"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05","Type":"ContainerStarted","Data":"c1c92b40b84844134674aadb4171371bb5f9e06b0a23c557be19df6afdffb50b"} Jul 11 07:42:53.646186 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.645995 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:53.646186 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.646059 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:42:53.647173 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.647146 2572 patch_prober.go:28] interesting pod/console-66548b89-wsnjw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body= Jul 11 07:42:53.647273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.647220 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-66548b89-wsnjw" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerName="console" probeResult="failure" output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" Jul 11 07:42:53.728924 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:53.728854 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg"] Jul 11 07:42:53.731595 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:53.731564 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33cbdf41_5531_448d_a358_636f510b7950.slice/crio-b51702ec27dc1dc915ce08fa1a09333541803fc098ab59e8423002f2bd768ecd WatchSource:0}: Error finding container b51702ec27dc1dc915ce08fa1a09333541803fc098ab59e8423002f2bd768ecd: Status 404 returned error can't find the container with id b51702ec27dc1dc915ce08fa1a09333541803fc098ab59e8423002f2bd768ecd Jul 11 07:42:54.644243 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:54.644203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" event={"ID":"e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05","Type":"ContainerStarted","Data":"2f04cba105eb02e39c8ae9345a7416bde0f0d837040199924f848d3f4c11249c"} Jul 11 07:42:54.645809 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:54.645779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" event={"ID":"33cbdf41-5531-448d-a358-636f510b7950","Type":"ContainerStarted","Data":"b51702ec27dc1dc915ce08fa1a09333541803fc098ab59e8423002f2bd768ecd"} Jul 11 07:42:54.660194 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:54.660146 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" podStartSLOduration=1.312924994 podStartE2EDuration="2.660130843s" totalImagesPullingTime="1.347205849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:54.658934775 +0000 UTC m=+264.504923669" watchObservedRunningTime="2026-07-11 07:42:54.660130843 +0000 UTC m=+264.506119735" Jul 11 07:42:55.651385 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:55.651338 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" event={"ID":"33cbdf41-5531-448d-a358-636f510b7950","Type":"ContainerStarted","Data":"731bd9810ddfc313e5944492a48b88df84420bc0dee87d5dad02eac9ec4476f6"} Jul 11 07:42:56.657137 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:56.657103 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" event={"ID":"33cbdf41-5531-448d-a358-636f510b7950","Type":"ContainerStarted","Data":"ccaae8b0ca85dda99dc93f2bcc9823afa0fd6ae403fd2fa652017f2d7d2c3eb6"} Jul 11 07:42:57.662728 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:57.662674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" event={"ID":"33cbdf41-5531-448d-a358-636f510b7950","Type":"ContainerStarted","Data":"2637bda469e04afc533eb0a8a2db8e05aca044cd6a1473eb2b64eedf9f2c188f"} Jul 11 07:42:57.685419 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:57.685358 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cfd4dfc88-wz4dg" podStartSLOduration=1.934302356 podStartE2EDuration="4.685340727s" totalImagesPullingTime="2.751038371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:57.684586636 +0000 UTC m=+267.530575552" watchObservedRunningTime="2026-07-11 07:42:57.685340727 +0000 UTC m=+267.531329620" Jul 11 07:42:58.943291 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:58.943256 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:42:58.972306 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:58.972273 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:42:58.975758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:58.975733 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:58.984624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:58.984592 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 11 07:42:58.988544 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:58.988518 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:42:59.061670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061630 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.061670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061667 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.061908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061729 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.061908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061818 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mclt6\" (UniqueName: \"kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.061908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061858 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.061908 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.062044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.061951 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.162957 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.162923 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.162971 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.162988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163049 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mclt6\" (UniqueName: \"kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163076 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163861 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.163972 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.163836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.164024 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.164004 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.164056 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.164038 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.165320 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.165298 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.165511 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.165493 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.170389 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.170367 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mclt6\" (UniqueName: \"kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6\") pod \"console-58dcdcffb-8wqg4\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.289443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.289409 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:42:59.299401 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.299374 2572 patch_prober.go:28] interesting pod/image-registry-856c5d9b5c-hgdfv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 11 07:42:59.299503 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.299426 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" podUID="f96658f9-74c3-42a4-aa52-6667cfe59c26" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 07:42:59.418889 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.418857 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:42:59.422442 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:42:59.422416 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c44e37_64f8_4cb8_bb89_128a921f9b3e.slice/crio-bc798160cd72db447c12fd58ad50548e89656142b7fb54d80a6a6d7b3e49fd62 WatchSource:0}: Error finding container bc798160cd72db447c12fd58ad50548e89656142b7fb54d80a6a6d7b3e49fd62: Status 404 returned error can't find the container with id bc798160cd72db447c12fd58ad50548e89656142b7fb54d80a6a6d7b3e49fd62 Jul 11 07:42:59.672128 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.672049 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58dcdcffb-8wqg4" event={"ID":"65c44e37-64f8-4cb8-bb89-128a921f9b3e","Type":"ContainerStarted","Data":"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc"} Jul 11 07:42:59.672128 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.672094 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58dcdcffb-8wqg4" event={"ID":"65c44e37-64f8-4cb8-bb89-128a921f9b3e","Type":"ContainerStarted","Data":"bc798160cd72db447c12fd58ad50548e89656142b7fb54d80a6a6d7b3e49fd62"} Jul 11 07:42:59.688234 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:42:59.688181 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58dcdcffb-8wqg4" podStartSLOduration=1.6881629 podStartE2EDuration="1.6881629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:42:59.686988199 +0000 UTC m=+269.532977152" watchObservedRunningTime="2026-07-11 07:42:59.6881629 +0000 UTC m=+269.534151792" Jul 11 07:43:01.573629 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:01.573596 2572 patch_prober.go:28] interesting pod/image-registry-856c5d9b5c-hgdfv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 11 07:43:01.574012 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:01.573649 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" podUID="f96658f9-74c3-42a4-aa52-6667cfe59c26" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 07:43:09.289994 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.289955 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:43:09.290450 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.290044 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:43:09.291641 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.291616 2572 patch_prober.go:28] interesting pod/console-58dcdcffb-8wqg4 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jul 11 07:43:09.291782 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.291661 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-58dcdcffb-8wqg4" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jul 11 07:43:09.298317 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.298286 2572 patch_prober.go:28] interesting pod/image-registry-856c5d9b5c-hgdfv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 11 07:43:09.298411 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:09.298325 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" podUID="f96658f9-74c3-42a4-aa52-6667cfe59c26" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 07:43:11.286061 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:43:11.286006 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qzl8f" podUID="27009180-8547-4131-8824-d912c8fe24a3" Jul 11 07:43:11.574041 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:11.573974 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-856c5d9b5c-hgdfv" Jul 11 07:43:11.724362 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:11.724334 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:12.697966 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:12.697929 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:43:12.698358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:12.698012 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:43:14.903126 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:14.903094 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:14.905330 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:14.905298 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27009180-8547-4131-8824-d912c8fe24a3-metrics-tls\") pod \"dns-default-qzl8f\" (UID: \"27009180-8547-4131-8824-d912c8fe24a3\") " pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:15.028019 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:15.027993 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbf8m\"" Jul 11 07:43:15.035983 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:15.035949 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:15.209239 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:15.209163 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qzl8f"] Jul 11 07:43:15.212235 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:43:15.212208 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27009180_8547_4131_8824_d912c8fe24a3.slice/crio-b188f74d4d5dca90f9208245eb56358c44b59d4f559eb09c97190a48f6408037 WatchSource:0}: Error finding container b188f74d4d5dca90f9208245eb56358c44b59d4f559eb09c97190a48f6408037: Status 404 returned error can't find the container with id b188f74d4d5dca90f9208245eb56358c44b59d4f559eb09c97190a48f6408037 Jul 11 07:43:15.741924 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:15.741887 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzl8f" event={"ID":"27009180-8547-4131-8824-d912c8fe24a3","Type":"ContainerStarted","Data":"b188f74d4d5dca90f9208245eb56358c44b59d4f559eb09c97190a48f6408037"} Jul 11 07:43:17.754969 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:17.754929 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzl8f" event={"ID":"27009180-8547-4131-8824-d912c8fe24a3","Type":"ContainerStarted","Data":"8f5198e2b4774db482a3a6dc539b40d9ae49933c7f9d6d05460e2f93567d170c"} Jul 11 07:43:17.755393 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:17.755009 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzl8f" event={"ID":"27009180-8547-4131-8824-d912c8fe24a3","Type":"ContainerStarted","Data":"307278dbcf71e9e53ec7e623cb2f773b9bcb82c9b1f485f123339431ade068f2"} Jul 11 07:43:17.755393 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:17.755046 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:17.771905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:17.771850 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qzl8f" podStartSLOduration=251.246172177 podStartE2EDuration="4m12.77183036s" totalImagesPullingTime="1.525658183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:43:17.770826664 +0000 UTC m=+287.616815575" watchObservedRunningTime="2026-07-11 07:43:17.77183036 +0000 UTC m=+287.617819309" Jul 11 07:43:19.293927 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:19.293898 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:43:19.298176 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:19.298151 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:43:23.963989 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:23.963925 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66548b89-wsnjw" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerName="console" containerID="cri-o://102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c" gracePeriod=15 Jul 11 07:43:24.200263 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.200240 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:43:24.388999 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.388964 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.388999 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389000 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.389255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389027 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.389255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389112 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.389255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389141 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksnv6\" (UniqueName: \"kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.389403 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389269 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config\") pod \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\" (UID: \"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e\") " Jul 11 07:43:24.389474 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389446 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:43:24.389533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389471 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca" (OuterVolumeSpecName: "service-ca") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:43:24.389582 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389571 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-oauth-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.389626 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389590 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.389626 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.389588 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config" (OuterVolumeSpecName: "console-config") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:43:24.391289 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.391269 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6" (OuterVolumeSpecName: "kube-api-access-ksnv6") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "kube-api-access-ksnv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:43:24.391558 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.391532 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:43:24.391558 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.391548 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" (UID: "9246ff50-06f2-47ef-b3f7-93e9bf21bb2e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:43:24.490340 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.490304 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.490340 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.490334 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksnv6\" (UniqueName: \"kubernetes.io/projected/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-kube-api-access-ksnv6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.490340 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.490347 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.490571 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.490354 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e-console-oauth-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:43:24.781808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.781767 2572 generic.go:356] "Generic (PLEG): container finished" podID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerID="102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c" exitCode=0 Jul 11 07:43:24.781998 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.781817 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66548b89-wsnjw" Jul 11 07:43:24.781998 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.781856 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66548b89-wsnjw" event={"ID":"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e","Type":"ContainerDied","Data":"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c"} Jul 11 07:43:24.781998 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.781919 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66548b89-wsnjw" event={"ID":"9246ff50-06f2-47ef-b3f7-93e9bf21bb2e","Type":"ContainerDied","Data":"afd662618b6c49a631f09288da4fd869f7dad3015ab75aebf146fd5a42ce747c"} Jul 11 07:43:24.781998 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.781945 2572 scope.go:117] "RemoveContainer" containerID="102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c" Jul 11 07:43:24.790203 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.790185 2572 scope.go:117] "RemoveContainer" containerID="102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c" Jul 11 07:43:24.790470 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:43:24.790445 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c\": container with ID starting with 102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c not found: ID does not exist" containerID="102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c" Jul 11 07:43:24.790520 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.790480 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c"} err="failed to get container status \"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c\": rpc error: code = NotFound desc = could not find container \"102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c\": container with ID starting with 102976cdda0b4a54956c2187ff6f292bd2b58a5acc8a5a4ce15e4b464dd4169c not found: ID does not exist" Jul 11 07:43:24.796684 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.796659 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:43:24.800008 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:24.799984 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66548b89-wsnjw"] Jul 11 07:43:26.709991 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:26.709955 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" path="/var/lib/kubelet/pods/9246ff50-06f2-47ef-b3f7-93e9bf21bb2e/volumes" Jul 11 07:43:27.760926 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:27.760896 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qzl8f" Jul 11 07:43:30.531231 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:30.531207 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:43:30.531661 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:30.531646 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:43:30.542414 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:30.542394 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 11 07:43:32.703487 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:32.703458 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:43:32.709802 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:32.709777 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-576994c976-gwgjv" Jul 11 07:43:41.849684 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:41.849645 2572 generic.go:356] "Generic (PLEG): container finished" podID="117717ba-3261-43a0-a194-5f464ce44c48" containerID="c731fd0aadd96cf4a0527f7b2866bdce7617c41caa21cbd99773fc99f0a29f7f" exitCode=0 Jul 11 07:43:41.850101 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:41.849733 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" event={"ID":"117717ba-3261-43a0-a194-5f464ce44c48","Type":"ContainerDied","Data":"c731fd0aadd96cf4a0527f7b2866bdce7617c41caa21cbd99773fc99f0a29f7f"} Jul 11 07:43:41.850101 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:41.850000 2572 scope.go:117] "RemoveContainer" containerID="c731fd0aadd96cf4a0527f7b2866bdce7617c41caa21cbd99773fc99f0a29f7f" Jul 11 07:43:42.854921 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:42.854888 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-k4dkb" event={"ID":"117717ba-3261-43a0-a194-5f464ce44c48","Type":"ContainerStarted","Data":"277d7afc28b53df00deaea7a2e441fd582841bc2689418306c071322af5ce63f"} Jul 11 07:43:51.227421 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:43:51.227389 2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c222b_66c5_423c_9cd1_699f17e9e22c.slice/crio-2fa7004214b9be510ad7d4b300eddd24467e63263e0d47a501a7c7a381a374fa.scope\": RecentStats: unable to find data in memory cache]" Jul 11 07:43:51.890907 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:51.890874 2572 generic.go:356] "Generic (PLEG): container finished" podID="358c222b-66c5-423c-9cd1-699f17e9e22c" containerID="2fa7004214b9be510ad7d4b300eddd24467e63263e0d47a501a7c7a381a374fa" exitCode=0 Jul 11 07:43:51.891074 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:51.890917 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" event={"ID":"358c222b-66c5-423c-9cd1-699f17e9e22c","Type":"ContainerDied","Data":"2fa7004214b9be510ad7d4b300eddd24467e63263e0d47a501a7c7a381a374fa"} Jul 11 07:43:51.891239 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:51.891226 2572 scope.go:117] "RemoveContainer" containerID="2fa7004214b9be510ad7d4b300eddd24467e63263e0d47a501a7c7a381a374fa" Jul 11 07:43:52.896716 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:43:52.896665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-r9jgt" event={"ID":"358c222b-66c5-423c-9cd1-699f17e9e22c","Type":"ContainerStarted","Data":"ad6ed5fa5daa3e5603f94640884a0fd71454ce338a4e5c61c9a8955c04cef642"} Jul 11 07:44:10.565367 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.565327 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 07:44:10.565849 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.565780 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerName="console" Jul 11 07:44:10.565849 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.565798 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerName="console" Jul 11 07:44:10.565963 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.565878 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9246ff50-06f2-47ef-b3f7-93e9bf21bb2e" containerName="console" Jul 11 07:44:10.570799 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.570778 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.573185 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.573162 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 11 07:44:10.573665 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.573642 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 11 07:44:10.573941 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.573919 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574824 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574866 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574876 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c7xgq\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574924 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574947 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 11 07:44:10.575060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.574883 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 11 07:44:10.579321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.578566 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 11 07:44:10.580968 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.580949 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 07:44:10.666212 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666212 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666210 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666408 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666232 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666408 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666251 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-web-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666408 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666315 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ltc\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-kube-api-access-c8ltc\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666408 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666366 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666528 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666411 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666528 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666466 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666528 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666485 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666528 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-out\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666642 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666543 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666642 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.666642 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.666588 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.767758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.767758 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767761 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767827 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767850 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767877 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-web-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767938 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ltc\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-kube-api-access-c8ltc\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768013 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.767963 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.768022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.768074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.768098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.768346 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.768126 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-out\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.771041 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771000 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771197 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb198836-f45a-4cb0-87c2-f4b07c62287f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771347 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-out\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771601 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771802 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.771991 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.772397 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.772180 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.775681 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.772518 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb198836-f45a-4cb0-87c2-f4b07c62287f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.775681 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.773195 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.775681 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.773322 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.775681 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.773512 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.778036 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.777804 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb198836-f45a-4cb0-87c2-f4b07c62287f-web-config\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.778495 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.778475 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ltc\" (UniqueName: \"kubernetes.io/projected/bb198836-f45a-4cb0-87c2-f4b07c62287f-kube-api-access-c8ltc\") pod \"alertmanager-main-0\" (UID: \"bb198836-f45a-4cb0-87c2-f4b07c62287f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:10.884199 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:10.884135 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 07:44:11.018749 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:11.018713 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 07:44:11.024318 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:44:11.024290 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb198836_f45a_4cb0_87c2_f4b07c62287f.slice/crio-05cb15b2a04ecf2b190ef7adf65639e55fb209ea8c6e5df245e17835cf51c659 WatchSource:0}: Error finding container 05cb15b2a04ecf2b190ef7adf65639e55fb209ea8c6e5df245e17835cf51c659: Status 404 returned error can't find the container with id 05cb15b2a04ecf2b190ef7adf65639e55fb209ea8c6e5df245e17835cf51c659 Jul 11 07:44:11.977240 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:11.977203 2572 generic.go:356] "Generic (PLEG): container finished" podID="bb198836-f45a-4cb0-87c2-f4b07c62287f" containerID="8485a6ee976643e643842c7ef75c3c4c01462dda35722659fcf7504aed24a5fe" exitCode=0 Jul 11 07:44:11.977595 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:11.977250 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerDied","Data":"8485a6ee976643e643842c7ef75c3c4c01462dda35722659fcf7504aed24a5fe"} Jul 11 07:44:11.977595 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:11.977319 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"05cb15b2a04ecf2b190ef7adf65639e55fb209ea8c6e5df245e17835cf51c659"} Jul 11 07:44:11.978349 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:11.978334 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 07:44:13.987434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:13.987405 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"dd9d9b419c0719990721bd2b0d67473c516d87cbe5f41ac8c218a32fbe0a9015"} Jul 11 07:44:13.987878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:13.987467 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"675063ad859b74f32f4390aedb6275a68f99e1f17d39ababbc72fa3b2383fd29"} Jul 11 07:44:13.987878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:13.987492 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"acfa73fbb88b78a086204814450fbf64e9452ac0835303890e62c744355bb6dd"} Jul 11 07:44:13.987878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:13.987515 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"21ded1418b00b12e556b140423f5f252c71b8d5684563e41ef9d288f801d1076"} Jul 11 07:44:13.987878 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:13.987539 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"b1e647de77b9db5798d55f247be9c8c934bdc4c40de93f8208bc1b0dec4c2f88"} Jul 11 07:44:14.994287 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:14.994251 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb198836-f45a-4cb0-87c2-f4b07c62287f","Type":"ContainerStarted","Data":"6517f554104be5b1d5d52ea0ad5c24983821a2e84c23e53508464dc3d85b07f8"} Jul 11 07:44:15.024423 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:15.024367 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.786385588 podStartE2EDuration="5.024347971s" totalImagesPullingTime="2.218906917s" totalInitContainerRuntime="19.055466ms" isStatefulPod=false podCreationTimestamp="2026-07-11 07:44:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:44:15.023938723 +0000 UTC m=+344.869927634" watchObservedRunningTime="2026-07-11 07:44:15.024347971 +0000 UTC m=+344.870336865" Jul 11 07:44:21.024930 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.024886 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:44:21.029533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.029507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.041817 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.041789 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:44:21.158159 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158166 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158232 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158281 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158316 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r248\" (UniqueName: \"kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158486 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158342 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.158486 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.158387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259368 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259539 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259389 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259539 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259539 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259464 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259539 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259539 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.259800 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.259573 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r248\" (UniqueName: \"kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.260255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.260223 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.260398 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.260264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.260398 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.260316 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.260398 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.260316 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.261873 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.261841 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.262014 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.261991 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.267151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.267133 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r248\" (UniqueName: \"kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248\") pod \"console-cc86d869-ncb4m\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.340182 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.340112 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:21.473784 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:21.473760 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:44:21.476456 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:44:21.476431 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae73e84_0ef2_4534_82a9_8f11bda5924f.slice/crio-f45c6f12c46cbbbf24d995026e798b6c1659f5c279bebb933d395a62e04e7fdc WatchSource:0}: Error finding container f45c6f12c46cbbbf24d995026e798b6c1659f5c279bebb933d395a62e04e7fdc: Status 404 returned error can't find the container with id f45c6f12c46cbbbf24d995026e798b6c1659f5c279bebb933d395a62e04e7fdc Jul 11 07:44:22.024239 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:22.024206 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc86d869-ncb4m" event={"ID":"8ae73e84-0ef2-4534-82a9-8f11bda5924f","Type":"ContainerStarted","Data":"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a"} Jul 11 07:44:22.024404 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:22.024251 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc86d869-ncb4m" event={"ID":"8ae73e84-0ef2-4534-82a9-8f11bda5924f","Type":"ContainerStarted","Data":"f45c6f12c46cbbbf24d995026e798b6c1659f5c279bebb933d395a62e04e7fdc"} Jul 11 07:44:22.041744 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:22.041676 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cc86d869-ncb4m" podStartSLOduration=1.041659382 podStartE2EDuration="1.041659382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:44:22.039961451 +0000 UTC m=+351.885950365" watchObservedRunningTime="2026-07-11 07:44:22.041659382 +0000 UTC m=+351.887648273" Jul 11 07:44:31.340282 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:31.340246 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:31.340672 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:31.340301 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:31.345358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:31.345338 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:32.068481 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:32.068449 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:44:32.113932 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:32.113901 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:44:57.137936 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.137885 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58dcdcffb-8wqg4" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerName="console" containerID="cri-o://5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc" gracePeriod=15 Jul 11 07:44:57.378007 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.377984 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:44:57.445624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445533 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445599 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mclt6\" (UniqueName: \"kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445626 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445678 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445816 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445870 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.445903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.445899 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config\") pod \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\" (UID: \"65c44e37-64f8-4cb8-bb89-128a921f9b3e\") " Jul 11 07:44:57.446107 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446062 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca" (OuterVolumeSpecName: "service-ca") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:44:57.446107 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446073 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:44:57.446207 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446191 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config" (OuterVolumeSpecName: "console-config") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:44:57.446276 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446250 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:44:57.446335 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446316 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-oauth-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.446335 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446331 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.446431 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446339 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.446431 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.446347 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c44e37-64f8-4cb8-bb89-128a921f9b3e-trusted-ca-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.447658 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.447631 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:44:57.447782 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.447726 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6" (OuterVolumeSpecName: "kube-api-access-mclt6") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "kube-api-access-mclt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:44:57.447901 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.447885 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "65c44e37-64f8-4cb8-bb89-128a921f9b3e" (UID: "65c44e37-64f8-4cb8-bb89-128a921f9b3e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:44:57.546900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.546871 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-oauth-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.546900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.546901 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c44e37-64f8-4cb8-bb89-128a921f9b3e-console-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:57.547134 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:57.546911 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mclt6\" (UniqueName: \"kubernetes.io/projected/65c44e37-64f8-4cb8-bb89-128a921f9b3e-kube-api-access-mclt6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:44:58.166385 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.166344 2572 generic.go:356] "Generic (PLEG): container finished" podID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerID="5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc" exitCode=0 Jul 11 07:44:58.166385 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.166386 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58dcdcffb-8wqg4" Jul 11 07:44:58.166859 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.166432 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58dcdcffb-8wqg4" event={"ID":"65c44e37-64f8-4cb8-bb89-128a921f9b3e","Type":"ContainerDied","Data":"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc"} Jul 11 07:44:58.166859 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.166482 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58dcdcffb-8wqg4" event={"ID":"65c44e37-64f8-4cb8-bb89-128a921f9b3e","Type":"ContainerDied","Data":"bc798160cd72db447c12fd58ad50548e89656142b7fb54d80a6a6d7b3e49fd62"} Jul 11 07:44:58.166859 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.166502 2572 scope.go:117] "RemoveContainer" containerID="5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc" Jul 11 07:44:58.175088 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.175071 2572 scope.go:117] "RemoveContainer" containerID="5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc" Jul 11 07:44:58.175303 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:44:58.175286 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc\": container with ID starting with 5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc not found: ID does not exist" containerID="5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc" Jul 11 07:44:58.175384 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.175311 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc"} err="failed to get container status \"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc\": rpc error: code = NotFound desc = could not find container \"5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc\": container with ID starting with 5ace729bc3b30a343b711f7e4fd898293c642b5a24b5099fceb2d566f2cab0dc not found: ID does not exist" Jul 11 07:44:58.185776 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.185748 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:44:58.189169 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.189149 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58dcdcffb-8wqg4"] Jul 11 07:44:58.709833 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:44:58.709801 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" path="/var/lib/kubelet/pods/65c44e37-64f8-4cb8-bb89-128a921f9b3e/volumes" Jul 11 07:45:01.513056 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.513020 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m"] Jul 11 07:45:01.513418 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.513347 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerName="console" Jul 11 07:45:01.513418 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.513357 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerName="console" Jul 11 07:45:01.513418 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.513412 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c44e37-64f8-4cb8-bb89-128a921f9b3e" containerName="console" Jul 11 07:45:01.515831 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.515816 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.518428 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.518397 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 07:45:01.518525 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.518484 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 07:45:01.519556 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.519543 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dlvcz\"" Jul 11 07:45:01.524336 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.524310 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m"] Jul 11 07:45:01.579635 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.579598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.579635 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.579639 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqdp2\" (UniqueName: \"kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.579884 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.579756 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.680913 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.680880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.681060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.680965 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.681060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.680990 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqdp2\" (UniqueName: \"kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.681263 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.681241 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.681334 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.681269 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.689184 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.689160 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqdp2\" (UniqueName: \"kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.825457 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.825381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:01.962008 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:01.961905 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m"] Jul 11 07:45:01.964374 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:01.964347 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda98472_6002_4d3a_bbc6_40992b2cf4fe.slice/crio-f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9 WatchSource:0}: Error finding container f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9: Status 404 returned error can't find the container with id f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9 Jul 11 07:45:02.184242 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:02.184157 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" event={"ID":"fda98472-6002-4d3a-bbc6-40992b2cf4fe","Type":"ContainerStarted","Data":"f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9"} Jul 11 07:45:09.216887 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:09.216851 2572 generic.go:356] "Generic (PLEG): container finished" podID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerID="e69cba7aa3c1f94a0fc291f05a81800263c4a8b955c19fa916c03553d3679e1b" exitCode=0 Jul 11 07:45:09.217274 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:09.216936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" event={"ID":"fda98472-6002-4d3a-bbc6-40992b2cf4fe","Type":"ContainerDied","Data":"e69cba7aa3c1f94a0fc291f05a81800263c4a8b955c19fa916c03553d3679e1b"} Jul 11 07:45:11.409365 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:45:11.409330 2572 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json848615802: no such file or directory" Jul 11 07:45:11.409732 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:45:11.409381 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get imageFs stats: missing image stats: " Jul 11 07:45:12.232199 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:12.232161 2572 generic.go:356] "Generic (PLEG): container finished" podID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerID="b1bf5d65626321f7e036d3fbe791ed2537822b0c9978ecf098e05f5a1936e1f8" exitCode=0 Jul 11 07:45:12.232375 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:12.232235 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" event={"ID":"fda98472-6002-4d3a-bbc6-40992b2cf4fe","Type":"ContainerDied","Data":"b1bf5d65626321f7e036d3fbe791ed2537822b0c9978ecf098e05f5a1936e1f8"} Jul 11 07:45:20.269397 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:20.269363 2572 generic.go:356] "Generic (PLEG): container finished" podID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerID="b095129854dfee0cb8f48a9d79afdb1e6b4f0688dde662a9fa31d0b3df1c43f1" exitCode=0 Jul 11 07:45:20.269798 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:20.269438 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" event={"ID":"fda98472-6002-4d3a-bbc6-40992b2cf4fe","Type":"ContainerDied","Data":"b095129854dfee0cb8f48a9d79afdb1e6b4f0688dde662a9fa31d0b3df1c43f1"} Jul 11 07:45:21.395256 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.395231 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:21.457378 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.457349 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqdp2\" (UniqueName: \"kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2\") pod \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " Jul 11 07:45:21.457532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.457386 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util\") pod \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " Jul 11 07:45:21.457532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.457443 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle\") pod \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\" (UID: \"fda98472-6002-4d3a-bbc6-40992b2cf4fe\") " Jul 11 07:45:21.458125 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.458099 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle" (OuterVolumeSpecName: "bundle") pod "fda98472-6002-4d3a-bbc6-40992b2cf4fe" (UID: "fda98472-6002-4d3a-bbc6-40992b2cf4fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:21.459444 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.459422 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2" (OuterVolumeSpecName: "kube-api-access-jqdp2") pod "fda98472-6002-4d3a-bbc6-40992b2cf4fe" (UID: "fda98472-6002-4d3a-bbc6-40992b2cf4fe"). InnerVolumeSpecName "kube-api-access-jqdp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:45:21.461393 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.461366 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util" (OuterVolumeSpecName: "util") pod "fda98472-6002-4d3a-bbc6-40992b2cf4fe" (UID: "fda98472-6002-4d3a-bbc6-40992b2cf4fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:21.558044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.557985 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqdp2\" (UniqueName: \"kubernetes.io/projected/fda98472-6002-4d3a-bbc6-40992b2cf4fe-kube-api-access-jqdp2\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:21.558044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.558008 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:21.558044 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:21.558018 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fda98472-6002-4d3a-bbc6-40992b2cf4fe-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:22.278927 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:22.278887 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" event={"ID":"fda98472-6002-4d3a-bbc6-40992b2cf4fe","Type":"ContainerDied","Data":"f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9"} Jul 11 07:45:22.278927 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:22.278908 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqf9m" Jul 11 07:45:22.279130 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:22.278957 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f70568f8997adeb23b7e5b8405bc204e97b2971ba4b4fdaf0d5e9813127867b9" Jul 11 07:45:29.446502 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.446461 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql"] Jul 11 07:45:29.447534 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447512 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="pull" Jul 11 07:45:29.447645 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447635 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="pull" Jul 11 07:45:29.447746 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447736 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="util" Jul 11 07:45:29.447822 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447814 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="util" Jul 11 07:45:29.447897 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447888 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="extract" Jul 11 07:45:29.447965 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.447955 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="extract" Jul 11 07:45:29.448133 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.448113 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fda98472-6002-4d3a-bbc6-40992b2cf4fe" containerName="extract" Jul 11 07:45:29.452025 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.452009 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.454982 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.454960 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:45:29.455065 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.454998 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 11 07:45:29.455065 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.455054 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5z8rk\"" Jul 11 07:45:29.462205 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.462181 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql"] Jul 11 07:45:29.522259 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.522222 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnxs\" (UniqueName: \"kubernetes.io/projected/986b21e9-dad9-4c56-b347-484ee8c2a260-kube-api-access-htnxs\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.522411 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.522280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986b21e9-dad9-4c56-b347-484ee8c2a260-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.623652 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.623608 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnxs\" (UniqueName: \"kubernetes.io/projected/986b21e9-dad9-4c56-b347-484ee8c2a260-kube-api-access-htnxs\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.623854 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.623689 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986b21e9-dad9-4c56-b347-484ee8c2a260-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.624056 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.624037 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986b21e9-dad9-4c56-b347-484ee8c2a260-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.631557 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.631533 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnxs\" (UniqueName: \"kubernetes.io/projected/986b21e9-dad9-4c56-b347-484ee8c2a260-kube-api-access-htnxs\") pod \"cert-manager-operator-controller-manager-55c444bf86-sr6ql\" (UID: \"986b21e9-dad9-4c56-b347-484ee8c2a260\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.762122 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.762083 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" Jul 11 07:45:29.891737 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:29.891556 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql"] Jul 11 07:45:29.894592 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:29.894565 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986b21e9_dad9_4c56_b347_484ee8c2a260.slice/crio-d7d7c3ea393822dae2431b62b805f424d628ed10eece426ca01afd35bb4fc0b4 WatchSource:0}: Error finding container d7d7c3ea393822dae2431b62b805f424d628ed10eece426ca01afd35bb4fc0b4: Status 404 returned error can't find the container with id d7d7c3ea393822dae2431b62b805f424d628ed10eece426ca01afd35bb4fc0b4 Jul 11 07:45:30.314300 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:30.314264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" event={"ID":"986b21e9-dad9-4c56-b347-484ee8c2a260","Type":"ContainerStarted","Data":"d7d7c3ea393822dae2431b62b805f424d628ed10eece426ca01afd35bb4fc0b4"} Jul 11 07:45:32.325895 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:32.325858 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" event={"ID":"986b21e9-dad9-4c56-b347-484ee8c2a260","Type":"ContainerStarted","Data":"63bd34b0bfdf255aaaaa31abf08dd00949b9c4c79e4392c6712771ebc8ba51aa"} Jul 11 07:45:32.346207 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:32.346160 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sr6ql" podStartSLOduration=1.5357353489999999 podStartE2EDuration="3.346145222s" totalImagesPullingTime="1.810409873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:45:32.343752871 +0000 UTC m=+422.189741788" watchObservedRunningTime="2026-07-11 07:45:32.346145222 +0000 UTC m=+422.192134113" Jul 11 07:45:33.226671 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.226637 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm"] Jul 11 07:45:33.229192 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.229168 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.231547 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.231529 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dlvcz\"" Jul 11 07:45:33.231715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.231686 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 07:45:33.232574 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.232559 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 07:45:33.239587 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.239563 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm"] Jul 11 07:45:33.257741 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.257694 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.257852 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.257756 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcbv\" (UniqueName: \"kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.257852 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.257785 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.358668 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.358641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcbv\" (UniqueName: \"kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.359027 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.358677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.359027 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.358801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.359167 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.359145 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.359202 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.359150 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.369637 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.369611 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcbv\" (UniqueName: \"kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.537603 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.537571 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:33.662651 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:33.662625 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm"] Jul 11 07:45:33.664448 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:33.664423 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a70281d_7bc1_4a5d_aba7_f0847c552362.slice/crio-9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4 WatchSource:0}: Error finding container 9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4: Status 404 returned error can't find the container with id 9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4 Jul 11 07:45:34.335312 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:34.335279 2572 generic.go:356] "Generic (PLEG): container finished" podID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerID="136ce27e212bda637052cde2d35ee190a093980203a0c499bdb2727fd8ed4104" exitCode=0 Jul 11 07:45:34.335489 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:34.335361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" event={"ID":"2a70281d-7bc1-4a5d-aba7-f0847c552362","Type":"ContainerDied","Data":"136ce27e212bda637052cde2d35ee190a093980203a0c499bdb2727fd8ed4104"} Jul 11 07:45:34.335489 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:34.335406 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" event={"ID":"2a70281d-7bc1-4a5d-aba7-f0847c552362","Type":"ContainerStarted","Data":"9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4"} Jul 11 07:45:36.345773 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:36.345683 2572 generic.go:356] "Generic (PLEG): container finished" podID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerID="f291d469277ca9923fc6df947aa06ac041c774567267c018a1389e393194c4b5" exitCode=0 Jul 11 07:45:36.345773 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:36.345748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" event={"ID":"2a70281d-7bc1-4a5d-aba7-f0847c552362","Type":"ContainerDied","Data":"f291d469277ca9923fc6df947aa06ac041c774567267c018a1389e393194c4b5"} Jul 11 07:45:37.352216 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:37.352178 2572 generic.go:356] "Generic (PLEG): container finished" podID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerID="2f06c58f337c5604d642f12e413edc5204090ccfa79d1379a87f5c794bf5addd" exitCode=0 Jul 11 07:45:37.352608 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:37.352255 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" event={"ID":"2a70281d-7bc1-4a5d-aba7-f0847c552362","Type":"ContainerDied","Data":"2f06c58f337c5604d642f12e413edc5204090ccfa79d1379a87f5c794bf5addd"} Jul 11 07:45:38.480042 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.480016 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:38.602184 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.602150 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util\") pod \"2a70281d-7bc1-4a5d-aba7-f0847c552362\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " Jul 11 07:45:38.602348 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.602216 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xcbv\" (UniqueName: \"kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv\") pod \"2a70281d-7bc1-4a5d-aba7-f0847c552362\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " Jul 11 07:45:38.602348 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.602292 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle\") pod \"2a70281d-7bc1-4a5d-aba7-f0847c552362\" (UID: \"2a70281d-7bc1-4a5d-aba7-f0847c552362\") " Jul 11 07:45:38.602693 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.602664 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle" (OuterVolumeSpecName: "bundle") pod "2a70281d-7bc1-4a5d-aba7-f0847c552362" (UID: "2a70281d-7bc1-4a5d-aba7-f0847c552362"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:38.604260 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.604233 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv" (OuterVolumeSpecName: "kube-api-access-2xcbv") pod "2a70281d-7bc1-4a5d-aba7-f0847c552362" (UID: "2a70281d-7bc1-4a5d-aba7-f0847c552362"). InnerVolumeSpecName "kube-api-access-2xcbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:45:38.607873 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.607827 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util" (OuterVolumeSpecName: "util") pod "2a70281d-7bc1-4a5d-aba7-f0847c552362" (UID: "2a70281d-7bc1-4a5d-aba7-f0847c552362"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:38.703656 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.703619 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:38.703656 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.703649 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a70281d-7bc1-4a5d-aba7-f0847c552362-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:38.703656 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:38.703661 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xcbv\" (UniqueName: \"kubernetes.io/projected/2a70281d-7bc1-4a5d-aba7-f0847c552362-kube-api-access-2xcbv\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:39.362496 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:39.362469 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" Jul 11 07:45:39.362496 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:39.362482 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwpvxm" event={"ID":"2a70281d-7bc1-4a5d-aba7-f0847c552362","Type":"ContainerDied","Data":"9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4"} Jul 11 07:45:39.362670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:39.362522 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e52cf4f675fefb80d8c828a279622c675556fb32bc2a551d89455d2d53b37c4" Jul 11 07:45:41.753573 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.753534 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7vtz"] Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754009 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="pull" Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754031 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="pull" Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754048 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="extract" Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754056 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="extract" Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754081 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="util" Jul 11 07:45:41.754096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754089 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="util" Jul 11 07:45:41.754385 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.754157 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a70281d-7bc1-4a5d-aba7-f0847c552362" containerName="extract" Jul 11 07:45:41.756092 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.756072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.758651 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.758629 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 11 07:45:41.758764 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.758629 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-spsb8\"" Jul 11 07:45:41.759654 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.759635 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 11 07:45:41.763929 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.763909 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7vtz"] Jul 11 07:45:41.831994 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.831959 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgpw2\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-kube-api-access-kgpw2\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.832160 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.832033 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.933218 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.933186 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgpw2\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-kube-api-access-kgpw2\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.933372 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.933255 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.941144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.941122 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgpw2\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-kube-api-access-kgpw2\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:41.941144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:41.941134 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2eec7b3-56b2-4569-a37c-2a087ba0cb25-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7vtz\" (UID: \"a2eec7b3-56b2-4569-a37c-2a087ba0cb25\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:42.075754 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:42.075655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:42.201259 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:42.201237 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7vtz"] Jul 11 07:45:42.203183 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:42.203151 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2eec7b3_56b2_4569_a37c_2a087ba0cb25.slice/crio-66274870da4e938821f1d80e9941eb74dce347f3e7ec3c4b19771bb5ec3db5e8 WatchSource:0}: Error finding container 66274870da4e938821f1d80e9941eb74dce347f3e7ec3c4b19771bb5ec3db5e8: Status 404 returned error can't find the container with id 66274870da4e938821f1d80e9941eb74dce347f3e7ec3c4b19771bb5ec3db5e8 Jul 11 07:45:42.376296 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:42.376216 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" event={"ID":"a2eec7b3-56b2-4569-a37c-2a087ba0cb25","Type":"ContainerStarted","Data":"66274870da4e938821f1d80e9941eb74dce347f3e7ec3c4b19771bb5ec3db5e8"} Jul 11 07:45:43.552401 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.552367 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg"] Jul 11 07:45:43.554964 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.554944 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.557517 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.557497 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 11 07:45:43.557622 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.557494 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 11 07:45:43.558655 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.558575 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-p5tvv\"" Jul 11 07:45:43.564936 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.564912 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg"] Jul 11 07:45:43.649461 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.649433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9889286-9663-4f42-8a69-af1213b94754-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.649615 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.649578 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6kf9\" (UniqueName: \"kubernetes.io/projected/c9889286-9663-4f42-8a69-af1213b94754-kube-api-access-f6kf9\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.750742 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.750682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6kf9\" (UniqueName: \"kubernetes.io/projected/c9889286-9663-4f42-8a69-af1213b94754-kube-api-access-f6kf9\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.750944 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.750792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9889286-9663-4f42-8a69-af1213b94754-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.751222 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.751193 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9889286-9663-4f42-8a69-af1213b94754-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.758623 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.758602 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6kf9\" (UniqueName: \"kubernetes.io/projected/c9889286-9663-4f42-8a69-af1213b94754-kube-api-access-f6kf9\") pod \"openshift-lws-operator-fd8ccff4c-k4kzg\" (UID: \"c9889286-9663-4f42-8a69-af1213b94754\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:43.866682 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:43.866595 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" Jul 11 07:45:44.823416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:44.823240 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg"] Jul 11 07:45:44.825761 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:44.825690 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9889286_9663_4f42_8a69_af1213b94754.slice/crio-935e1650d3ace464a11f9810da91cf083843a00459a127da303df09af1d545e7 WatchSource:0}: Error finding container 935e1650d3ace464a11f9810da91cf083843a00459a127da303df09af1d545e7: Status 404 returned error can't find the container with id 935e1650d3ace464a11f9810da91cf083843a00459a127da303df09af1d545e7 Jul 11 07:45:45.396229 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:45.396188 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" event={"ID":"a2eec7b3-56b2-4569-a37c-2a087ba0cb25","Type":"ContainerStarted","Data":"41bc3b8f3978816722d3485a584541f93d807a055a7fd4e05e2fce5914a71879"} Jul 11 07:45:45.396420 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:45.396258 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:45.397503 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:45.397480 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" event={"ID":"c9889286-9663-4f42-8a69-af1213b94754","Type":"ContainerStarted","Data":"935e1650d3ace464a11f9810da91cf083843a00459a127da303df09af1d545e7"} Jul 11 07:45:45.411991 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:45.411924 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" podStartSLOduration=1.886470927 podStartE2EDuration="4.411907015s" totalImagesPullingTime="2.525436088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:45:45.411352483 +0000 UTC m=+435.257341373" watchObservedRunningTime="2026-07-11 07:45:45.411907015 +0000 UTC m=+435.257895908" Jul 11 07:45:47.410293 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:47.410261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" event={"ID":"c9889286-9663-4f42-8a69-af1213b94754","Type":"ContainerStarted","Data":"56365dd1d96719a0a17b37c459214197161ece8c367c83303068e9c304dacc99"} Jul 11 07:45:47.425920 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:47.425875 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k4kzg" podStartSLOduration=2.79997281 podStartE2EDuration="4.425860307s" totalImagesPullingTime="1.625887497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:45:47.423563998 +0000 UTC m=+437.269552910" watchObservedRunningTime="2026-07-11 07:45:47.425860307 +0000 UTC m=+437.271849199" Jul 11 07:45:50.181949 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.181910 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4"] Jul 11 07:45:50.208163 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.208137 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4"] Jul 11 07:45:50.208320 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.208230 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.210938 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.210915 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 07:45:50.211133 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.211120 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dlvcz\"" Jul 11 07:45:50.212059 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.212042 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 07:45:50.314550 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.314514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.314756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.314589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8wg\" (UniqueName: \"kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.314756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.314611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.415336 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.415299 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8wg\" (UniqueName: \"kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.415336 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.415340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.415532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.415376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.415738 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.415723 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.415795 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.415767 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.423400 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.423374 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8wg\" (UniqueName: \"kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.517384 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.517355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:50.668545 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:50.668526 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4"] Jul 11 07:45:50.673360 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:45:50.673332 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93202516_a154_4672_a4a7_52d3c153874f.slice/crio-96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6 WatchSource:0}: Error finding container 96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6: Status 404 returned error can't find the container with id 96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6 Jul 11 07:45:51.406564 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:51.406537 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7vtz" Jul 11 07:45:51.430756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:51.430724 2572 generic.go:356] "Generic (PLEG): container finished" podID="93202516-a154-4672-a4a7-52d3c153874f" containerID="d24fe8984f8a53288ff86bf04d65bff2dbf974d9e792d3aa77de1c63fa7bc8e4" exitCode=0 Jul 11 07:45:51.430905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:51.430806 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" event={"ID":"93202516-a154-4672-a4a7-52d3c153874f","Type":"ContainerDied","Data":"d24fe8984f8a53288ff86bf04d65bff2dbf974d9e792d3aa77de1c63fa7bc8e4"} Jul 11 07:45:51.430905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:51.430870 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" event={"ID":"93202516-a154-4672-a4a7-52d3c153874f","Type":"ContainerStarted","Data":"96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6"} Jul 11 07:45:52.438194 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:52.438101 2572 generic.go:356] "Generic (PLEG): container finished" podID="93202516-a154-4672-a4a7-52d3c153874f" containerID="fb1cc662e8c913c3302d509e3cf34d2f7bcebfa43863873e7b26e147cdc95c31" exitCode=0 Jul 11 07:45:52.438194 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:52.438157 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" event={"ID":"93202516-a154-4672-a4a7-52d3c153874f","Type":"ContainerDied","Data":"fb1cc662e8c913c3302d509e3cf34d2f7bcebfa43863873e7b26e147cdc95c31"} Jul 11 07:45:53.444867 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:53.444833 2572 generic.go:356] "Generic (PLEG): container finished" podID="93202516-a154-4672-a4a7-52d3c153874f" containerID="cad00524c77ff37c6dde7f4a7a1899923d80571d1613306ff9d37141e70308f7" exitCode=0 Jul 11 07:45:53.445242 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:53.444880 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" event={"ID":"93202516-a154-4672-a4a7-52d3c153874f","Type":"ContainerDied","Data":"cad00524c77ff37c6dde7f4a7a1899923d80571d1613306ff9d37141e70308f7"} Jul 11 07:45:54.582905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.582877 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:54.652778 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.652744 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util\") pod \"93202516-a154-4672-a4a7-52d3c153874f\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " Jul 11 07:45:54.652940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.652823 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8wg\" (UniqueName: \"kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg\") pod \"93202516-a154-4672-a4a7-52d3c153874f\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " Jul 11 07:45:54.652940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.652844 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle\") pod \"93202516-a154-4672-a4a7-52d3c153874f\" (UID: \"93202516-a154-4672-a4a7-52d3c153874f\") " Jul 11 07:45:54.653638 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.653615 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle" (OuterVolumeSpecName: "bundle") pod "93202516-a154-4672-a4a7-52d3c153874f" (UID: "93202516-a154-4672-a4a7-52d3c153874f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:54.654936 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.654913 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg" (OuterVolumeSpecName: "kube-api-access-ql8wg") pod "93202516-a154-4672-a4a7-52d3c153874f" (UID: "93202516-a154-4672-a4a7-52d3c153874f"). InnerVolumeSpecName "kube-api-access-ql8wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:45:54.657886 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.657848 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util" (OuterVolumeSpecName: "util") pod "93202516-a154-4672-a4a7-52d3c153874f" (UID: "93202516-a154-4672-a4a7-52d3c153874f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:45:54.753845 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.753812 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8wg\" (UniqueName: \"kubernetes.io/projected/93202516-a154-4672-a4a7-52d3c153874f-kube-api-access-ql8wg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:54.753845 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.753839 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:54.753845 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:54.753848 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93202516-a154-4672-a4a7-52d3c153874f-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:45:55.455910 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:55.455874 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" Jul 11 07:45:55.455910 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:55.455894 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvl4h4" event={"ID":"93202516-a154-4672-a4a7-52d3c153874f","Type":"ContainerDied","Data":"96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6"} Jul 11 07:45:55.456123 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:45:55.455937 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96b0440c1d9f9f47bb90f3ad031cda2c93cc1f7b90ab08a4bb319e9430a8d2e6" Jul 11 07:46:08.068535 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.068497 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2"] Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.068984 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="pull" Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069003 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="pull" Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069020 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="extract" Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069029 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="extract" Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069069 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="util" Jul 11 07:46:08.069121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069078 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="util" Jul 11 07:46:08.069415 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.069159 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="93202516-a154-4672-a4a7-52d3c153874f" containerName="extract" Jul 11 07:46:08.076650 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.076629 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.079785 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.079767 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 11 07:46:08.079894 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.079876 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 11 07:46:08.080035 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.080019 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 11 07:46:08.080119 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.080095 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n26xj\"" Jul 11 07:46:08.080390 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.080371 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 11 07:46:08.105284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.105247 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2"] Jul 11 07:46:08.173295 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.173258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.173461 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.173305 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.173461 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.173379 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.173461 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.173412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jr4l\" (UniqueName: \"kubernetes.io/projected/e3d45bfc-4a26-45d5-985a-83d019112f88-kube-api-access-5jr4l\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.173461 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.173449 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274110 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274164 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jr4l\" (UniqueName: \"kubernetes.io/projected/e3d45bfc-4a26-45d5-985a-83d019112f88-kube-api-access-5jr4l\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274273 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274192 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.274632 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.274611 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-manifests\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.276484 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.276460 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d45bfc-4a26-45d5-985a-83d019112f88-tmp\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.276682 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.276657 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.276747 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.276715 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e3d45bfc-4a26-45d5-985a-83d019112f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.282973 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.282955 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jr4l\" (UniqueName: \"kubernetes.io/projected/e3d45bfc-4a26-45d5-985a-83d019112f88-kube-api-access-5jr4l\") pod \"opendatahub-operator-controller-manager-78d8b67ddd-5czv2\" (UID: \"e3d45bfc-4a26-45d5-985a-83d019112f88\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.386931 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.386839 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:08.533383 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:08.533353 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2"] Jul 11 07:46:08.536649 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:46:08.536618 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d45bfc_4a26_45d5_985a_83d019112f88.slice/crio-6f5eb9745fb71f7d18b992836629f7e140bdda6b1e902862ea49a4998550bc8e WatchSource:0}: Error finding container 6f5eb9745fb71f7d18b992836629f7e140bdda6b1e902862ea49a4998550bc8e: Status 404 returned error can't find the container with id 6f5eb9745fb71f7d18b992836629f7e140bdda6b1e902862ea49a4998550bc8e Jul 11 07:46:09.522426 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:09.522394 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" event={"ID":"e3d45bfc-4a26-45d5-985a-83d019112f88","Type":"ContainerStarted","Data":"6f5eb9745fb71f7d18b992836629f7e140bdda6b1e902862ea49a4998550bc8e"} Jul 11 07:46:12.538117 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:12.538081 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" event={"ID":"e3d45bfc-4a26-45d5-985a-83d019112f88","Type":"ContainerStarted","Data":"6e19be3ec5474e25bb149da2cbac8d3654dd45d4927a91ebe46cb9c4c9b9a9ff"} Jul 11 07:46:13.544395 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:13.544359 2572 generic.go:356] "Generic (PLEG): container finished" podID="e3d45bfc-4a26-45d5-985a-83d019112f88" containerID="6e19be3ec5474e25bb149da2cbac8d3654dd45d4927a91ebe46cb9c4c9b9a9ff" exitCode=0 Jul 11 07:46:13.544798 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:13.544432 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" event={"ID":"e3d45bfc-4a26-45d5-985a-83d019112f88","Type":"ContainerDied","Data":"6e19be3ec5474e25bb149da2cbac8d3654dd45d4927a91ebe46cb9c4c9b9a9ff"} Jul 11 07:46:14.550336 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:14.550302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" event={"ID":"e3d45bfc-4a26-45d5-985a-83d019112f88","Type":"ContainerStarted","Data":"09acfbdda02535a6264dab2724380a8cfefdf1ee98d1a4426a33961becedc324"} Jul 11 07:46:14.550696 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:14.550438 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:14.585641 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:14.585584 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" podStartSLOduration=2.179685913 podStartE2EDuration="6.585566915s" totalImagesPullingTime="3.332360414s" totalInitContainerRuntime="1.073520588s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:14.585176364 +0000 UTC m=+464.431165283" watchObservedRunningTime="2026-07-11 07:46:14.585566915 +0000 UTC m=+464.431555821" Jul 11 07:46:25.258712 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.258672 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b"] Jul 11 07:46:25.262295 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.262278 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.266404 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.266381 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 11 07:46:25.266404 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.266399 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4gpvv\"" Jul 11 07:46:25.266587 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.266416 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 11 07:46:25.266587 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.266387 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 11 07:46:25.275956 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.275918 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b"] Jul 11 07:46:25.318619 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.318584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-manager-config\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.318808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.318740 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27lvj\" (UniqueName: \"kubernetes.io/projected/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-kube-api-access-27lvj\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.318808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.318776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.318808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.318793 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-metrics-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.419945 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.419907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27lvj\" (UniqueName: \"kubernetes.io/projected/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-kube-api-access-27lvj\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.420115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.419955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.420115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.419971 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-metrics-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.420115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.419992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-manager-config\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.420634 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.420610 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-manager-config\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.422436 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.422413 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-metrics-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.422533 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.422476 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-cert\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.428694 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.428662 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27lvj\" (UniqueName: \"kubernetes.io/projected/a3b09c8e-52c2-4a06-bcb4-357f666dd2e1-kube-api-access-27lvj\") pod \"lws-controller-manager-697d57cd68-kr86b\" (UID: \"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1\") " pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.557776 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.557674 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8b67ddd-5czv2" Jul 11 07:46:25.571859 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.571826 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:25.705051 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:25.705026 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b"] Jul 11 07:46:25.707203 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:46:25.707176 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b09c8e_52c2_4a06_bcb4_357f666dd2e1.slice/crio-afc9a71916e60c4085d0566efab8f80286bb8016e34235d2d883fb9ad62e9e33 WatchSource:0}: Error finding container afc9a71916e60c4085d0566efab8f80286bb8016e34235d2d883fb9ad62e9e33: Status 404 returned error can't find the container with id afc9a71916e60c4085d0566efab8f80286bb8016e34235d2d883fb9ad62e9e33 Jul 11 07:46:26.609914 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:26.609881 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" event={"ID":"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1","Type":"ContainerStarted","Data":"afc9a71916e60c4085d0566efab8f80286bb8016e34235d2d883fb9ad62e9e33"} Jul 11 07:46:27.617191 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:27.617091 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" event={"ID":"a3b09c8e-52c2-4a06-bcb4-357f666dd2e1","Type":"ContainerStarted","Data":"221ce14a27a90c0bc034aef874f26730ce2368d5923f144cec94b4624d218251"} Jul 11 07:46:27.617191 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:27.617188 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:27.643006 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:27.642954 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" podStartSLOduration=1.049772585 podStartE2EDuration="2.642936371s" totalImagesPullingTime="1.593163786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:27.642419474 +0000 UTC m=+477.488408366" watchObservedRunningTime="2026-07-11 07:46:27.642936371 +0000 UTC m=+477.488925263" Jul 11 07:46:28.869021 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.868990 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjls2"] Jul 11 07:46:28.872459 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.872440 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:28.874998 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.874976 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 11 07:46:28.882089 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.882065 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjls2"] Jul 11 07:46:28.952201 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.952166 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nffv8\" (UniqueName: \"kubernetes.io/projected/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-kube-api-access-nffv8\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:28.952201 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:28.952213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.053448 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.053413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nffv8\" (UniqueName: \"kubernetes.io/projected/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-kube-api-access-nffv8\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.053641 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.053474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.053641 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:46:29.053624 2572 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 11 07:46:29.053799 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:46:29.053722 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs podName:d92e977b-a6f6-4b3d-92b9-6d6bd33aab07 nodeName:}" failed. No retries permitted until 2026-07-11 07:46:29.553678479 +0000 UTC m=+479.399667355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs") pod "model-serving-api-54d89bd5fc-zjls2" (UID: "d92e977b-a6f6-4b3d-92b9-6d6bd33aab07") : secret "model-serving-api-tls" not found Jul 11 07:46:29.064322 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.064297 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nffv8\" (UniqueName: \"kubernetes.io/projected/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-kube-api-access-nffv8\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.558888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.558853 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.561556 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.561530 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92e977b-a6f6-4b3d-92b9-6d6bd33aab07-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjls2\" (UID: \"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.783284 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.783242 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:29.914428 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:29.914400 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjls2"] Jul 11 07:46:29.916834 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:46:29.916803 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92e977b_a6f6_4b3d_92b9_6d6bd33aab07.slice/crio-9da48657fdbf899e0c0ecab57d7cd726bed68ccbc7edf28201174ac6b97d46c3 WatchSource:0}: Error finding container 9da48657fdbf899e0c0ecab57d7cd726bed68ccbc7edf28201174ac6b97d46c3: Status 404 returned error can't find the container with id 9da48657fdbf899e0c0ecab57d7cd726bed68ccbc7edf28201174ac6b97d46c3 Jul 11 07:46:30.632762 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:30.632726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" event={"ID":"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07","Type":"ContainerStarted","Data":"9da48657fdbf899e0c0ecab57d7cd726bed68ccbc7edf28201174ac6b97d46c3"} Jul 11 07:46:32.643492 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:32.643453 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" event={"ID":"d92e977b-a6f6-4b3d-92b9-6d6bd33aab07","Type":"ContainerStarted","Data":"2cfc47c2e1747f422fd84e16cd62a1202c0e3627fefdedc6dc519fd781f54b22"} Jul 11 07:46:32.643892 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:32.643634 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:46:32.659572 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:32.659520 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" podStartSLOduration=2.973049028 podStartE2EDuration="4.659504051s" totalImagesPullingTime="1.686455023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:32.658268115 +0000 UTC m=+482.504257053" watchObservedRunningTime="2026-07-11 07:46:32.659504051 +0000 UTC m=+482.505492943" Jul 11 07:46:34.611176 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.611141 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr"] Jul 11 07:46:34.614553 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.614536 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.617137 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.617121 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 11 07:46:34.617228 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.617167 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mh52f\"" Jul 11 07:46:34.629121 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.629097 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr"] Jul 11 07:46:34.704263 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tdx6\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-kube-api-access-7tdx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704279 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/890ddd9d-34a6-477a-95a2-4cf5b7284464-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704304 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704399 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704451 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704431 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704602 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704602 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704526 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.704602 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.704576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805080 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805036 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tdx6\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-kube-api-access-7tdx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805096 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/890ddd9d-34a6-477a-95a2-4cf5b7284464-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805192 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805253 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805253 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805558 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805412 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805558 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805476 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805672 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805646 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805803 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805767 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805926 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805810 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.805926 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805838 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.806006 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.805960 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/890ddd9d-34a6-477a-95a2-4cf5b7284464-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.807617 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.807585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.807760 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.807745 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.816135 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.816115 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.817060 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.817031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tdx6\" (UniqueName: \"kubernetes.io/projected/890ddd9d-34a6-477a-95a2-4cf5b7284464-kube-api-access-7tdx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf426bdr\" (UID: \"890ddd9d-34a6-477a-95a2-4cf5b7284464\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:34.925519 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:34.925420 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:35.063037 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:35.063007 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr"] Jul 11 07:46:35.066231 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:46:35.066207 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890ddd9d_34a6_477a_95a2_4cf5b7284464.slice/crio-5848cbd8531512fdfd41a2a26de6c6718157d9d38222274b05389269d25304c8 WatchSource:0}: Error finding container 5848cbd8531512fdfd41a2a26de6c6718157d9d38222274b05389269d25304c8: Status 404 returned error can't find the container with id 5848cbd8531512fdfd41a2a26de6c6718157d9d38222274b05389269d25304c8 Jul 11 07:46:35.660537 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:35.660493 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" event={"ID":"890ddd9d-34a6-477a-95a2-4cf5b7284464","Type":"ContainerStarted","Data":"5848cbd8531512fdfd41a2a26de6c6718157d9d38222274b05389269d25304c8"} Jul 11 07:46:37.556251 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.556206 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 07:46:37.556531 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.556288 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 07:46:37.556531 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.556321 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 07:46:37.672952 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.672910 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" event={"ID":"890ddd9d-34a6-477a-95a2-4cf5b7284464","Type":"ContainerStarted","Data":"123fbb311f8ec7282abbfc78a96e8e69b9caca677d6cc23a6cd481b23ba5a20e"} Jul 11 07:46:37.724321 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.724265 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" podStartSLOduration=1.236630766 podStartE2EDuration="3.724247314s" totalImagesPullingTime="2.487616548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:37.723902626 +0000 UTC m=+487.569891519" watchObservedRunningTime="2026-07-11 07:46:37.724247314 +0000 UTC m=+487.570236206" Jul 11 07:46:37.887388 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.887310 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-klwhr"] Jul 11 07:46:37.890600 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.890583 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:37.893599 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.893566 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mr5bs\"" Jul 11 07:46:37.894045 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.894027 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 11 07:46:37.902757 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.902731 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-klwhr"] Jul 11 07:46:37.926301 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:37.926268 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:38.039062 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.039026 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnbwz\" (UniqueName: \"kubernetes.io/projected/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-kube-api-access-lnbwz\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.039255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.039107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-cert\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.140358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.140262 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-cert\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.140358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.140324 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnbwz\" (UniqueName: \"kubernetes.io/projected/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-kube-api-access-lnbwz\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.142691 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.142671 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-cert\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.154240 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.154210 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnbwz\" (UniqueName: \"kubernetes.io/projected/2ffd435a-6d4e-4e17-b12e-2e1c217aa80e-kube-api-access-lnbwz\") pod \"kserve-controller-manager-d6f4b65b8-klwhr\" (UID: \"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.202047 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.202012 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:38.349879 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.349794 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-klwhr"] Jul 11 07:46:38.351976 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:46:38.351925 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ffd435a_6d4e_4e17_b12e_2e1c217aa80e.slice/crio-4d2473eb5bf9d09b9c442502c73ca7975a6767013d38f9acb1b4c9ef33ebeeaa WatchSource:0}: Error finding container 4d2473eb5bf9d09b9c442502c73ca7975a6767013d38f9acb1b4c9ef33ebeeaa: Status 404 returned error can't find the container with id 4d2473eb5bf9d09b9c442502c73ca7975a6767013d38f9acb1b4c9ef33ebeeaa Jul 11 07:46:38.624666 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.624632 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-697d57cd68-kr86b" Jul 11 07:46:38.680137 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.680070 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" event={"ID":"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e","Type":"ContainerStarted","Data":"4d2473eb5bf9d09b9c442502c73ca7975a6767013d38f9acb1b4c9ef33ebeeaa"} Jul 11 07:46:38.932069 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:38.931965 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:39.685647 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:39.685614 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:39.686722 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:39.686679 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf426bdr" Jul 11 07:46:40.691672 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:40.691621 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" event={"ID":"2ffd435a-6d4e-4e17-b12e-2e1c217aa80e","Type":"ContainerStarted","Data":"4743f57a5ff4f567f0565f51e0892a3572451ab1140817b5689c9d53d6a2cf69"} Jul 11 07:46:40.692133 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:40.691694 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:46:40.713722 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:40.712188 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" podStartSLOduration=1.531054736 podStartE2EDuration="3.712163445s" totalImagesPullingTime="2.181108709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:46:40.708085347 +0000 UTC m=+490.554074241" watchObservedRunningTime="2026-07-11 07:46:40.712163445 +0000 UTC m=+490.558152343" Jul 11 07:46:43.652808 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:46:43.652780 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zjls2" Jul 11 07:47:11.701905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:11.701875 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-klwhr" Jul 11 07:47:44.043365 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.043336 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk"] Jul 11 07:47:44.047778 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.047762 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.050532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.050506 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 11 07:47:44.050664 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.050533 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 11 07:47:44.050769 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.050755 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6pgs\"" Jul 11 07:47:44.057236 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.057210 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk"] Jul 11 07:47:44.216962 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.216869 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.216962 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.216917 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqpg\" (UniqueName: \"kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.217324 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.216987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.318110 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.318027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.318110 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.318105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.318308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.318136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqpg\" (UniqueName: \"kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.318471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.318450 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.318471 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.318463 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.326681 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.326663 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqpg\" (UniqueName: \"kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.357173 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.357153 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:44.447754 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.447723 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm"] Jul 11 07:47:44.452378 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.452355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.460530 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.460459 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm"] Jul 11 07:47:44.489619 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.489591 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk"] Jul 11 07:47:44.492435 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:47:44.492408 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f920617_30e7_41f8_9795_567a04711e31.slice/crio-d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1 WatchSource:0}: Error finding container d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1: Status 404 returned error can't find the container with id d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1 Jul 11 07:47:44.620865 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.620839 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.620973 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.620904 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.620973 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.620936 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvjcw\" (UniqueName: \"kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.721841 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.721816 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.721986 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.721864 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvjcw\" (UniqueName: \"kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.721986 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.721937 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.722147 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.722126 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.722238 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.722221 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.729317 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.729293 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvjcw\" (UniqueName: \"kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.765844 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.765823 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:44.894307 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:44.894275 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm"] Jul 11 07:47:44.896088 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:47:44.896068 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de989c1_0185_4c33_bd31_2279c6002bcf.slice/crio-91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634 WatchSource:0}: Error finding container 91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634: Status 404 returned error can't find the container with id 91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634 Jul 11 07:47:45.007628 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.007598 2572 generic.go:356] "Generic (PLEG): container finished" podID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerID="d775537acaa1b768e73cad10e59ba315ca2a1e4a3097f37b14ac802febcbb9d7" exitCode=0 Jul 11 07:47:45.007746 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.007680 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" event={"ID":"7de989c1-0185-4c33-bd31-2279c6002bcf","Type":"ContainerDied","Data":"d775537acaa1b768e73cad10e59ba315ca2a1e4a3097f37b14ac802febcbb9d7"} Jul 11 07:47:45.007814 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.007744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" event={"ID":"7de989c1-0185-4c33-bd31-2279c6002bcf","Type":"ContainerStarted","Data":"91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634"} Jul 11 07:47:45.009219 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.009198 2572 generic.go:356] "Generic (PLEG): container finished" podID="7f920617-30e7-41f8-9795-567a04711e31" containerID="5ed76174131a0fd760ce94708c2a6e74a0313fa705c0990c3eae6f8526d68553" exitCode=0 Jul 11 07:47:45.009434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.009251 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" event={"ID":"7f920617-30e7-41f8-9795-567a04711e31","Type":"ContainerDied","Data":"5ed76174131a0fd760ce94708c2a6e74a0313fa705c0990c3eae6f8526d68553"} Jul 11 07:47:45.009434 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.009285 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" event={"ID":"7f920617-30e7-41f8-9795-567a04711e31","Type":"ContainerStarted","Data":"d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1"} Jul 11 07:47:45.044107 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.044083 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz"] Jul 11 07:47:45.047570 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.047555 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.055421 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.055402 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz"] Jul 11 07:47:45.227673 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.227610 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvgv\" (UniqueName: \"kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.227809 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.227669 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.227809 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.227768 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.328984 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.328957 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.329107 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.328998 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvgv\" (UniqueName: \"kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.329151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.329085 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.329355 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.329339 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.329413 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.329397 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.337545 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.337516 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvgv\" (UniqueName: \"kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.372359 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.372337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:45.498506 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.498475 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz"] Jul 11 07:47:45.499566 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:47:45.499540 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f52ce27_bcf2_48fd_9ed6_96b47cf77a06.slice/crio-877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734 WatchSource:0}: Error finding container 877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734: Status 404 returned error can't find the container with id 877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734 Jul 11 07:47:45.652777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.652746 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4"] Jul 11 07:47:45.656258 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.656230 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.664498 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.664469 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4"] Jul 11 07:47:45.835195 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.835163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prk6c\" (UniqueName: \"kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.835358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.835225 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.835358 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.835284 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.936028 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.936008 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prk6c\" (UniqueName: \"kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.936133 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.936046 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.936133 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.936068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.936366 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.936343 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.936422 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.936392 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.943806 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.943783 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prk6c\" (UniqueName: \"kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:45.967993 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:45.967963 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:46.019104 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.018584 2572 generic.go:356] "Generic (PLEG): container finished" podID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerID="c792bfabd7a3c55e07b9e58f7227032840930072e1af31b908d7eac8f946ff03" exitCode=0 Jul 11 07:47:46.019104 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.018674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" event={"ID":"7de989c1-0185-4c33-bd31-2279c6002bcf","Type":"ContainerDied","Data":"c792bfabd7a3c55e07b9e58f7227032840930072e1af31b908d7eac8f946ff03"} Jul 11 07:47:46.021369 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.021330 2572 generic.go:356] "Generic (PLEG): container finished" podID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerID="ce54edca0511b31dbbfa676085cced0f6758327ec676d34c9c2c5ab4f62d98ef" exitCode=0 Jul 11 07:47:46.021717 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.021618 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" event={"ID":"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06","Type":"ContainerDied","Data":"ce54edca0511b31dbbfa676085cced0f6758327ec676d34c9c2c5ab4f62d98ef"} Jul 11 07:47:46.021717 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.021671 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" event={"ID":"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06","Type":"ContainerStarted","Data":"877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734"} Jul 11 07:47:46.026721 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.026378 2572 generic.go:356] "Generic (PLEG): container finished" podID="7f920617-30e7-41f8-9795-567a04711e31" containerID="3eb2095ae893e6b4e71a3f73ca8fbcb96791f36f2b603fe7294d5b2cde836f81" exitCode=0 Jul 11 07:47:46.026721 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.026466 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" event={"ID":"7f920617-30e7-41f8-9795-567a04711e31","Type":"ContainerDied","Data":"3eb2095ae893e6b4e71a3f73ca8fbcb96791f36f2b603fe7294d5b2cde836f81"} Jul 11 07:47:46.107880 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:46.107854 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4"] Jul 11 07:47:46.109835 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:47:46.109808 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845005a9_54c4_43ef_8ce3_9d622ea894b7.slice/crio-416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af WatchSource:0}: Error finding container 416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af: Status 404 returned error can't find the container with id 416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af Jul 11 07:47:47.033784 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.033756 2572 generic.go:356] "Generic (PLEG): container finished" podID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerID="53d92a3346a1c5cc17be36f3ec4d3be31c4e67e3b79634a256a87edff43a351e" exitCode=0 Jul 11 07:47:47.033956 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.033843 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" event={"ID":"7de989c1-0185-4c33-bd31-2279c6002bcf","Type":"ContainerDied","Data":"53d92a3346a1c5cc17be36f3ec4d3be31c4e67e3b79634a256a87edff43a351e"} Jul 11 07:47:47.035492 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.035469 2572 generic.go:356] "Generic (PLEG): container finished" podID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerID="d4dc16ea7417e47fb6016042a9467bf98a85c29733be7b199459b77c95153c1f" exitCode=0 Jul 11 07:47:47.035592 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.035505 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" event={"ID":"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06","Type":"ContainerDied","Data":"d4dc16ea7417e47fb6016042a9467bf98a85c29733be7b199459b77c95153c1f"} Jul 11 07:47:47.037084 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.037035 2572 generic.go:356] "Generic (PLEG): container finished" podID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerID="13be65146c4a4f13d8f18bed16768693d98aa94a25ca7269f5ba9fe2d85b57ea" exitCode=0 Jul 11 07:47:47.037084 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.037067 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" event={"ID":"845005a9-54c4-43ef-8ce3-9d622ea894b7","Type":"ContainerDied","Data":"13be65146c4a4f13d8f18bed16768693d98aa94a25ca7269f5ba9fe2d85b57ea"} Jul 11 07:47:47.037235 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.037113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" event={"ID":"845005a9-54c4-43ef-8ce3-9d622ea894b7","Type":"ContainerStarted","Data":"416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af"} Jul 11 07:47:47.039436 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.039412 2572 generic.go:356] "Generic (PLEG): container finished" podID="7f920617-30e7-41f8-9795-567a04711e31" containerID="6435465b26e779327b6fba1e7bba11ea3e9784764ed0752fe1f40347808bdf4c" exitCode=0 Jul 11 07:47:47.039518 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:47.039499 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" event={"ID":"7f920617-30e7-41f8-9795-567a04711e31","Type":"ContainerDied","Data":"6435465b26e779327b6fba1e7bba11ea3e9784764ed0752fe1f40347808bdf4c"} Jul 11 07:47:48.046447 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.046375 2572 generic.go:356] "Generic (PLEG): container finished" podID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerID="e862a939443e0c55f9057f2310871e7c21f01dd9cdc2829e45d1c58cbf3493b5" exitCode=0 Jul 11 07:47:48.046447 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.046443 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" event={"ID":"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06","Type":"ContainerDied","Data":"e862a939443e0c55f9057f2310871e7c21f01dd9cdc2829e45d1c58cbf3493b5"} Jul 11 07:47:48.048238 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.048213 2572 generic.go:356] "Generic (PLEG): container finished" podID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerID="3aecf9df0bb9795d74909c2f0ac8a9af56e2bf818494feda4f417b606adf0d5f" exitCode=0 Jul 11 07:47:48.048334 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.048299 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" event={"ID":"845005a9-54c4-43ef-8ce3-9d622ea894b7","Type":"ContainerDied","Data":"3aecf9df0bb9795d74909c2f0ac8a9af56e2bf818494feda4f417b606adf0d5f"} Jul 11 07:47:48.188895 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.188867 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:48.256443 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.256421 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:48.262947 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.262926 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbqpg\" (UniqueName: \"kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg\") pod \"7f920617-30e7-41f8-9795-567a04711e31\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " Jul 11 07:47:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.262959 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle\") pod \"7f920617-30e7-41f8-9795-567a04711e31\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " Jul 11 07:47:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.262981 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvjcw\" (UniqueName: \"kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw\") pod \"7de989c1-0185-4c33-bd31-2279c6002bcf\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " Jul 11 07:47:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.263019 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util\") pod \"7f920617-30e7-41f8-9795-567a04711e31\" (UID: \"7f920617-30e7-41f8-9795-567a04711e31\") " Jul 11 07:47:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.263038 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util\") pod \"7de989c1-0185-4c33-bd31-2279c6002bcf\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " Jul 11 07:47:48.263055 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.263054 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle\") pod \"7de989c1-0185-4c33-bd31-2279c6002bcf\" (UID: \"7de989c1-0185-4c33-bd31-2279c6002bcf\") " Jul 11 07:47:48.263474 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.263445 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle" (OuterVolumeSpecName: "bundle") pod "7f920617-30e7-41f8-9795-567a04711e31" (UID: "7f920617-30e7-41f8-9795-567a04711e31"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:48.263779 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.263754 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle" (OuterVolumeSpecName: "bundle") pod "7de989c1-0185-4c33-bd31-2279c6002bcf" (UID: "7de989c1-0185-4c33-bd31-2279c6002bcf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:48.265222 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.265197 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw" (OuterVolumeSpecName: "kube-api-access-wvjcw") pod "7de989c1-0185-4c33-bd31-2279c6002bcf" (UID: "7de989c1-0185-4c33-bd31-2279c6002bcf"). InnerVolumeSpecName "kube-api-access-wvjcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:47:48.265309 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.265264 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg" (OuterVolumeSpecName: "kube-api-access-jbqpg") pod "7f920617-30e7-41f8-9795-567a04711e31" (UID: "7f920617-30e7-41f8-9795-567a04711e31"). InnerVolumeSpecName "kube-api-access-jbqpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:47:48.268516 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.268496 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util" (OuterVolumeSpecName: "util") pod "7de989c1-0185-4c33-bd31-2279c6002bcf" (UID: "7de989c1-0185-4c33-bd31-2279c6002bcf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:48.269082 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.269065 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util" (OuterVolumeSpecName: "util") pod "7f920617-30e7-41f8-9795-567a04711e31" (UID: "7f920617-30e7-41f8-9795-567a04711e31"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364099 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364122 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364130 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de989c1-0185-4c33-bd31-2279c6002bcf-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364139 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbqpg\" (UniqueName: \"kubernetes.io/projected/7f920617-30e7-41f8-9795-567a04711e31-kube-api-access-jbqpg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364149 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f920617-30e7-41f8-9795-567a04711e31-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:48.364168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:48.364157 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvjcw\" (UniqueName: \"kubernetes.io/projected/7de989c1-0185-4c33-bd31-2279c6002bcf-kube-api-access-wvjcw\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:49.055292 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.055267 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" Jul 11 07:47:49.055693 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.055264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm" event={"ID":"7de989c1-0185-4c33-bd31-2279c6002bcf","Type":"ContainerDied","Data":"91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634"} Jul 11 07:47:49.055693 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.055390 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91466eeb6c3ad605e9b746e1561a7beb175fbdb3a5b0989c3f1f667f0a293634" Jul 11 07:47:49.057609 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.057583 2572 generic.go:356] "Generic (PLEG): container finished" podID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerID="6b502b61a6acbfd025507ac34242a775d626bf53d96132ac1978ee6094f33133" exitCode=0 Jul 11 07:47:49.057698 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.057685 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" event={"ID":"845005a9-54c4-43ef-8ce3-9d622ea894b7","Type":"ContainerDied","Data":"6b502b61a6acbfd025507ac34242a775d626bf53d96132ac1978ee6094f33133"} Jul 11 07:47:49.059532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.059515 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" Jul 11 07:47:49.059610 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.059554 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk" event={"ID":"7f920617-30e7-41f8-9795-567a04711e31","Type":"ContainerDied","Data":"d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1"} Jul 11 07:47:49.059610 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.059589 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d54d3ecf3a05c69d60a926b86c1740464bb4e4261b6a1c7740de216597bb24d1" Jul 11 07:47:49.193243 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.193221 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:49.272230 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.272205 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzvgv\" (UniqueName: \"kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv\") pod \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " Jul 11 07:47:49.272378 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.272264 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle\") pod \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " Jul 11 07:47:49.272378 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.272298 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util\") pod \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\" (UID: \"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06\") " Jul 11 07:47:49.272803 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.272777 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle" (OuterVolumeSpecName: "bundle") pod "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" (UID: "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:49.274251 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.274228 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv" (OuterVolumeSpecName: "kube-api-access-dzvgv") pod "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" (UID: "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06"). InnerVolumeSpecName "kube-api-access-dzvgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:47:49.277786 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.277749 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util" (OuterVolumeSpecName: "util") pod "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" (UID: "4f52ce27-bcf2-48fd-9ed6-96b47cf77a06"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:49.373143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.373080 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzvgv\" (UniqueName: \"kubernetes.io/projected/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-kube-api-access-dzvgv\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:49.373143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.373106 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:49.373143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:49.373120 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f52ce27-bcf2-48fd-9ed6-96b47cf77a06-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:50.066279 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.066244 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" Jul 11 07:47:50.066738 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.066278 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz" event={"ID":"4f52ce27-bcf2-48fd-9ed6-96b47cf77a06","Type":"ContainerDied","Data":"877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734"} Jul 11 07:47:50.066738 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.066347 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="877394ed0524c8b0f80c6d2e8a824ff3bf40fdcd855cd9569e71bdc3b23c7734" Jul 11 07:47:50.198483 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.198459 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:50.280691 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.280655 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util\") pod \"845005a9-54c4-43ef-8ce3-9d622ea894b7\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " Jul 11 07:47:50.280865 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.280697 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle\") pod \"845005a9-54c4-43ef-8ce3-9d622ea894b7\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " Jul 11 07:47:50.280865 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.280780 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prk6c\" (UniqueName: \"kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c\") pod \"845005a9-54c4-43ef-8ce3-9d622ea894b7\" (UID: \"845005a9-54c4-43ef-8ce3-9d622ea894b7\") " Jul 11 07:47:50.281172 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.281126 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle" (OuterVolumeSpecName: "bundle") pod "845005a9-54c4-43ef-8ce3-9d622ea894b7" (UID: "845005a9-54c4-43ef-8ce3-9d622ea894b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:50.282771 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.282750 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c" (OuterVolumeSpecName: "kube-api-access-prk6c") pod "845005a9-54c4-43ef-8ce3-9d622ea894b7" (UID: "845005a9-54c4-43ef-8ce3-9d622ea894b7"). InnerVolumeSpecName "kube-api-access-prk6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:47:50.288137 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.288099 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util" (OuterVolumeSpecName: "util") pod "845005a9-54c4-43ef-8ce3-9d622ea894b7" (UID: "845005a9-54c4-43ef-8ce3-9d622ea894b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:47:50.382308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.382249 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prk6c\" (UniqueName: \"kubernetes.io/projected/845005a9-54c4-43ef-8ce3-9d622ea894b7-kube-api-access-prk6c\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:50.382308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.382271 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-util\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:50.382308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:50.382279 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845005a9-54c4-43ef-8ce3-9d622ea894b7-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:47:51.073332 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:51.073304 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" Jul 11 07:47:51.073690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:51.073304 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4" event={"ID":"845005a9-54c4-43ef-8ce3-9d622ea894b7","Type":"ContainerDied","Data":"416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af"} Jul 11 07:47:51.073690 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:47:51.073424 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="416029470e141a2adbc8aefb173e299b71cfd3f490653744b1f70e4c406403af" Jul 11 07:48:05.808278 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808242 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr"] Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808582 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808591 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808599 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808606 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808618 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808623 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="extract" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808633 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="util" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808638 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="util" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808644 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="pull" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808648 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="pull" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808658 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="util" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808672 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="util" Jul 11 07:48:05.808670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808679 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808684 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808691 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="extract" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808696 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="extract" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808718 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="util" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808724 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="util" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808730 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808734 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808740 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="util" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808744 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="util" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808750 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808755 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="pull" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808825 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="845005a9-54c4-43ef-8ce3-9d622ea894b7" containerName="extract" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808832 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f52ce27-bcf2-48fd-9ed6-96b47cf77a06" containerName="extract" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808838 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de989c1-0185-4c33-bd31-2279c6002bcf" containerName="extract" Jul 11 07:48:05.809136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.808843 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f920617-30e7-41f8-9795-567a04711e31" containerName="extract" Jul 11 07:48:05.813353 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.813334 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:05.815993 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.815971 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f5qqf\"" Jul 11 07:48:05.816106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.815974 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 11 07:48:05.816106 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.815974 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 11 07:48:05.823432 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.823409 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr"] Jul 11 07:48:05.914037 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.913995 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:05.914168 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:05.914093 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh5lz\" (UniqueName: \"kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.014836 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.014799 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.014977 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.014902 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh5lz\" (UniqueName: \"kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.015176 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.015156 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.023244 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.023211 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh5lz\" (UniqueName: \"kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.124008 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.123920 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:06.253890 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:06.253865 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr"] Jul 11 07:48:06.255413 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:48:06.255387 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693d8a6c_5b23_4490_8f07_6c25731de85a.slice/crio-b711a0bf0d5b51b6573c0cc2ced6af270f93464ffc5b5f4be4070c62f0f3b957 WatchSource:0}: Error finding container b711a0bf0d5b51b6573c0cc2ced6af270f93464ffc5b5f4be4070c62f0f3b957: Status 404 returned error can't find the container with id b711a0bf0d5b51b6573c0cc2ced6af270f93464ffc5b5f4be4070c62f0f3b957 Jul 11 07:48:07.157774 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:07.157734 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" event={"ID":"693d8a6c-5b23-4490-8f07-6c25731de85a","Type":"ContainerStarted","Data":"b711a0bf0d5b51b6573c0cc2ced6af270f93464ffc5b5f4be4070c62f0f3b957"} Jul 11 07:48:13.196272 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:13.196228 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" event={"ID":"693d8a6c-5b23-4490-8f07-6c25731de85a","Type":"ContainerStarted","Data":"139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57"} Jul 11 07:48:13.196622 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:13.196296 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:13.219757 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:13.219695 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" podStartSLOduration=2.18695031 podStartE2EDuration="8.219676465s" totalImagesPullingTime="6.032726155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:13.218065836 +0000 UTC m=+583.064054728" watchObservedRunningTime="2026-07-11 07:48:13.219676465 +0000 UTC m=+583.065665358" Jul 11 07:48:13.579601 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:13.579565 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:48:24.205038 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:24.205010 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:25.593905 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.593869 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj"] Jul 11 07:48:25.597416 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.597394 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.612615 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.612579 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj"] Jul 11 07:48:25.715933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.715895 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbt8\" (UniqueName: \"kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.715933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.715938 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.816611 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.816572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.816823 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.816772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbt8\" (UniqueName: \"kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.817065 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.817043 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.836092 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.836064 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbt8\" (UniqueName: \"kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8\") pod \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.867608 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.867538 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj"] Jul 11 07:48:25.867869 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.867856 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:25.874517 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.874493 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj"] Jul 11 07:48:25.910144 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.910111 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr"] Jul 11 07:48:25.910308 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.910293 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" containerName="manager" containerID="cri-o://139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57" gracePeriod=2 Jul 11 07:48:25.928108 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.928082 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr"] Jul 11 07:48:25.928108 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:25.928097 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"693d8a6c-5b23-4490-8f07-6c25731de85a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\": pods \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:25.942606 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.942577 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:25.942963 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.942950 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" containerName="manager" Jul 11 07:48:25.943024 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.942965 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" containerName="manager" Jul 11 07:48:25.943059 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.943044 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" containerName="manager" Jul 11 07:48:25.946112 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.946084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:25.948992 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:25.948954 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:25.961986 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:25.961952 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:26.119305 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.119208 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtsh2\" (UniqueName: \"kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.119458 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.119311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.148888 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.148864 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:26.151194 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:26.151168 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:26.220096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.220064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtsh2\" (UniqueName: \"kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.220260 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.220148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.220502 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.220484 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.230609 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.230585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtsh2\" (UniqueName: \"kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fcmwh\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.267675 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.267631 2572 generic.go:356] "Generic (PLEG): container finished" podID="693d8a6c-5b23-4490-8f07-6c25731de85a" containerID="139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57" exitCode=0 Jul 11 07:48:26.267893 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.267678 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:26.267893 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.267757 2572 scope.go:117] "RemoveContainer" containerID="139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57" Jul 11 07:48:26.270445 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:26.270418 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:26.276484 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.276463 2572 scope.go:117] "RemoveContainer" containerID="139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57" Jul 11 07:48:26.276869 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:26.276843 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57\": container with ID starting with 139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57 not found: ID does not exist" containerID="139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57" Jul 11 07:48:26.276926 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.276879 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57"} err="failed to get container status \"139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57\": rpc error: code = NotFound desc = could not find container \"139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57\": container with ID starting with 139cc72f3580ef40e9ee37b9e7ddb65d6537fa9a0b20bf25babbe3a24459bb57 not found: ID does not exist" Jul 11 07:48:26.305117 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.305081 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:26.321521 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.321494 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume\") pod \"693d8a6c-5b23-4490-8f07-6c25731de85a\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " Jul 11 07:48:26.321600 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.321538 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh5lz\" (UniqueName: \"kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz\") pod \"693d8a6c-5b23-4490-8f07-6c25731de85a\" (UID: \"693d8a6c-5b23-4490-8f07-6c25731de85a\") " Jul 11 07:48:26.321985 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.321949 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "693d8a6c-5b23-4490-8f07-6c25731de85a" (UID: "693d8a6c-5b23-4490-8f07-6c25731de85a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:48:26.323468 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.323442 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz" (OuterVolumeSpecName: "kube-api-access-wh5lz") pod "693d8a6c-5b23-4490-8f07-6c25731de85a" (UID: "693d8a6c-5b23-4490-8f07-6c25731de85a"). InnerVolumeSpecName "kube-api-access-wh5lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:48:26.422306 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.422273 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/693d8a6c-5b23-4490-8f07-6c25731de85a-extensions-socket-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:26.422306 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.422299 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh5lz\" (UniqueName: \"kubernetes.io/projected/693d8a6c-5b23-4490-8f07-6c25731de85a-kube-api-access-wh5lz\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:26.444588 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.444564 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:26.447822 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:48:26.447797 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c5a1ca_d462_4569_9932_d02769533478.slice/crio-5d0ddac667e7bb579d7cf7370d689ed90119469cb66212fd116d28e00a6e5199 WatchSource:0}: Error finding container 5d0ddac667e7bb579d7cf7370d689ed90119469cb66212fd116d28e00a6e5199: Status 404 returned error can't find the container with id 5d0ddac667e7bb579d7cf7370d689ed90119469cb66212fd116d28e00a6e5199 Jul 11 07:48:26.577640 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:26.577609 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hxxpr\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hxxpr" Jul 11 07:48:26.711954 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:26.711876 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="693d8a6c-5b23-4490-8f07-6c25731de85a" path="/var/lib/kubelet/pods/693d8a6c-5b23-4490-8f07-6c25731de85a/volumes" Jul 11 07:48:27.274586 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:27.274546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" event={"ID":"e7c5a1ca-d462-4569-9932-d02769533478","Type":"ContainerStarted","Data":"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925"} Jul 11 07:48:27.274830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:27.274735 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:27.274830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:27.274785 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" event={"ID":"e7c5a1ca-d462-4569-9932-d02769533478","Type":"ContainerStarted","Data":"5d0ddac667e7bb579d7cf7370d689ed90119469cb66212fd116d28e00a6e5199"} Jul 11 07:48:27.297787 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:27.297736 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" podStartSLOduration=2.297720582 podStartE2EDuration="2.297720582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:27.294373029 +0000 UTC m=+597.140361936" watchObservedRunningTime="2026-07-11 07:48:27.297720582 +0000 UTC m=+597.143709467" Jul 11 07:48:28.413081 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:28.413036 2572 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-lk9jj_kuadrant-system_54d56fc7-8ec1-421f-8ee5-30376ddc7a77_0(c1edbc2e93da7d2b510e4cb03f94f36fb5fd64e3d272676d9cd9c951ea9be624): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-lk9jj to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 11 07:48:28.413380 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:28.413105 2572 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-lk9jj_kuadrant-system_54d56fc7-8ec1-421f-8ee5-30376ddc7a77_0(c1edbc2e93da7d2b510e4cb03f94f36fb5fd64e3d272676d9cd9c951ea9be624): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-lk9jj to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:29.286566 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.286534 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:29.298297 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.298275 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:29.300558 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:29.300531 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="54d56fc7-8ec1-421f-8ee5-30376ddc7a77" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:29.348033 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.348004 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume\") pod \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " Jul 11 07:48:29.348187 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.348053 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmbt8\" (UniqueName: \"kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8\") pod \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\" (UID: \"54d56fc7-8ec1-421f-8ee5-30376ddc7a77\") " Jul 11 07:48:29.348292 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.348270 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "54d56fc7-8ec1-421f-8ee5-30376ddc7a77" (UID: "54d56fc7-8ec1-421f-8ee5-30376ddc7a77"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:48:29.350096 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.350070 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8" (OuterVolumeSpecName: "kube-api-access-xmbt8") pod "54d56fc7-8ec1-421f-8ee5-30376ddc7a77" (UID: "54d56fc7-8ec1-421f-8ee5-30376ddc7a77"). InnerVolumeSpecName "kube-api-access-xmbt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:48:29.449376 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.449341 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-extensions-socket-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:29.449376 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:29.449368 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmbt8\" (UniqueName: \"kubernetes.io/projected/54d56fc7-8ec1-421f-8ee5-30376ddc7a77-kube-api-access-xmbt8\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:30.291699 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:30.291659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:30.294189 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:30.294157 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="54d56fc7-8ec1-421f-8ee5-30376ddc7a77" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:30.300141 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:30.300110 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="54d56fc7-8ec1-421f-8ee5-30376ddc7a77" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:30.567630 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:30.567549 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:48:30.568039 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:30.568021 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:48:30.710339 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:30.710294 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lk9jj\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="54d56fc7-8ec1-421f-8ee5-30376ddc7a77" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lk9jj" Jul 11 07:48:30.711875 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:30.711848 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54d56fc7-8ec1-421f-8ee5-30376ddc7a77" path="/var/lib/kubelet/pods/54d56fc7-8ec1-421f-8ee5-30376ddc7a77/volumes" Jul 11 07:48:38.284082 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.284050 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:38.601848 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.601745 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cc86d869-ncb4m" podUID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" containerName="console" containerID="cri-o://481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a" gracePeriod=15 Jul 11 07:48:38.842785 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.842761 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:48:38.933617 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933534 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933617 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933576 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933617 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933604 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933928 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933750 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933928 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933832 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933928 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933872 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.933928 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933917 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r248\" (UniqueName: \"kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248\") pod \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\" (UID: \"8ae73e84-0ef2-4534-82a9-8f11bda5924f\") " Jul 11 07:48:38.934143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.933921 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:48:38.934143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934039 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca" (OuterVolumeSpecName: "service-ca") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:48:38.934143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934045 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config" (OuterVolumeSpecName: "console-config") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:48:38.934143 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934128 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 07:48:38.934289 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934281 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:38.934330 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934293 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:38.934330 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934302 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-trusted-ca-bundle\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:38.934330 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.934310 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ae73e84-0ef2-4534-82a9-8f11bda5924f-oauth-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:38.936001 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.935977 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:48:38.936001 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.935994 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248" (OuterVolumeSpecName: "kube-api-access-9r248") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "kube-api-access-9r248". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:48:38.936163 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:38.936032 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8ae73e84-0ef2-4534-82a9-8f11bda5924f" (UID: "8ae73e84-0ef2-4534-82a9-8f11bda5924f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:48:39.035047 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.035019 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-oauth-config\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:39.035047 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.035044 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ae73e84-0ef2-4534-82a9-8f11bda5924f-console-serving-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:39.035047 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.035054 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r248\" (UniqueName: \"kubernetes.io/projected/8ae73e84-0ef2-4534-82a9-8f11bda5924f-kube-api-access-9r248\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:39.337360 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.337325 2572 generic.go:356] "Generic (PLEG): container finished" podID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" containerID="481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a" exitCode=0 Jul 11 07:48:39.337777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.337370 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc86d869-ncb4m" Jul 11 07:48:39.337777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.337409 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc86d869-ncb4m" event={"ID":"8ae73e84-0ef2-4534-82a9-8f11bda5924f","Type":"ContainerDied","Data":"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a"} Jul 11 07:48:39.337777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.337465 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc86d869-ncb4m" event={"ID":"8ae73e84-0ef2-4534-82a9-8f11bda5924f","Type":"ContainerDied","Data":"f45c6f12c46cbbbf24d995026e798b6c1659f5c279bebb933d395a62e04e7fdc"} Jul 11 07:48:39.337777 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.337491 2572 scope.go:117] "RemoveContainer" containerID="481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a" Jul 11 07:48:39.346854 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.346834 2572 scope.go:117] "RemoveContainer" containerID="481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a" Jul 11 07:48:39.347100 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:39.347083 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a\": container with ID starting with 481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a not found: ID does not exist" containerID="481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a" Jul 11 07:48:39.347170 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.347107 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a"} err="failed to get container status \"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a\": rpc error: code = NotFound desc = could not find container \"481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a\": container with ID starting with 481265bc66a64dc53a3995ecd8de664a59d193b73f289c814941f5faec703e5a not found: ID does not exist" Jul 11 07:48:39.360087 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.360063 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:48:39.368145 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:39.366109 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cc86d869-ncb4m"] Jul 11 07:48:40.712059 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:40.712028 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" path="/var/lib/kubelet/pods/8ae73e84-0ef2-4534-82a9-8f11bda5924f/volumes" Jul 11 07:48:41.688051 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.688021 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:41.688247 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.688205 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" podUID="e7c5a1ca-d462-4569-9932-d02769533478" containerName="manager" containerID="cri-o://952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925" gracePeriod=10 Jul 11 07:48:41.874426 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.874096 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:48:41.874895 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.874660 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" containerName="console" Jul 11 07:48:41.874895 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.874680 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" containerName="console" Jul 11 07:48:41.874895 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.874811 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ae73e84-0ef2-4534-82a9-8f11bda5924f" containerName="console" Jul 11 07:48:41.877194 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.877170 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:41.889398 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.889371 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:48:41.939824 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.939767 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:41.959613 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.959584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:41.959613 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:41.959613 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-754w5\" (UniqueName: \"kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.060427 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060395 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtsh2\" (UniqueName: \"kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2\") pod \"e7c5a1ca-d462-4569-9932-d02769533478\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " Jul 11 07:48:42.060427 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060427 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume\") pod \"e7c5a1ca-d462-4569-9932-d02769533478\" (UID: \"e7c5a1ca-d462-4569-9932-d02769533478\") " Jul 11 07:48:42.060670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.060670 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060573 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-754w5\" (UniqueName: \"kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.060933 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060907 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7c5a1ca-d462-4569-9932-d02769533478" (UID: "e7c5a1ca-d462-4569-9932-d02769533478"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:48:42.060995 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.060942 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.062418 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.062398 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2" (OuterVolumeSpecName: "kube-api-access-rtsh2") pod "e7c5a1ca-d462-4569-9932-d02769533478" (UID: "e7c5a1ca-d462-4569-9932-d02769533478"). InnerVolumeSpecName "kube-api-access-rtsh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:48:42.068715 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.068686 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-754w5\" (UniqueName: \"kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5\") pod \"kuadrant-operator-controller-manager-86878c769f-p5lqm\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.161254 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.161221 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtsh2\" (UniqueName: \"kubernetes.io/projected/e7c5a1ca-d462-4569-9932-d02769533478-kube-api-access-rtsh2\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:42.161254 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.161246 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c5a1ca-d462-4569-9932-d02769533478-extensions-socket-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:48:42.189136 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.189113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:42.328289 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.328266 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:48:42.330278 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:48:42.330256 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e511a88_69fe_46a4_ab7b_f0aaa15c7545.slice/crio-41e18e71cfd2d21da4f17140ffc502c8c54d96b888a4111f52affcbc7449c708 WatchSource:0}: Error finding container 41e18e71cfd2d21da4f17140ffc502c8c54d96b888a4111f52affcbc7449c708: Status 404 returned error can't find the container with id 41e18e71cfd2d21da4f17140ffc502c8c54d96b888a4111f52affcbc7449c708 Jul 11 07:48:42.355004 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.354974 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" event={"ID":"3e511a88-69fe-46a4-ab7b-f0aaa15c7545","Type":"ContainerStarted","Data":"41e18e71cfd2d21da4f17140ffc502c8c54d96b888a4111f52affcbc7449c708"} Jul 11 07:48:42.356250 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.356231 2572 generic.go:356] "Generic (PLEG): container finished" podID="e7c5a1ca-d462-4569-9932-d02769533478" containerID="952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925" exitCode=0 Jul 11 07:48:42.356309 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.356273 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" Jul 11 07:48:42.356366 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.356272 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" event={"ID":"e7c5a1ca-d462-4569-9932-d02769533478","Type":"ContainerDied","Data":"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925"} Jul 11 07:48:42.356418 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.356399 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh" event={"ID":"e7c5a1ca-d462-4569-9932-d02769533478","Type":"ContainerDied","Data":"5d0ddac667e7bb579d7cf7370d689ed90119469cb66212fd116d28e00a6e5199"} Jul 11 07:48:42.356468 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.356436 2572 scope.go:117] "RemoveContainer" containerID="952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925" Jul 11 07:48:42.368641 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.368624 2572 scope.go:117] "RemoveContainer" containerID="952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925" Jul 11 07:48:42.368971 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:48:42.368953 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925\": container with ID starting with 952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925 not found: ID does not exist" containerID="952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925" Jul 11 07:48:42.369050 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.368978 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925"} err="failed to get container status \"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925\": rpc error: code = NotFound desc = could not find container \"952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925\": container with ID starting with 952a646e58c66e649da77d70d006d6d8dbeb891e078bc14760aadc79bb036925 not found: ID does not exist" Jul 11 07:48:42.382923 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.382906 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:42.388216 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.388195 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fcmwh"] Jul 11 07:48:42.717864 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:42.713109 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c5a1ca-d462-4569-9932-d02769533478" path="/var/lib/kubelet/pods/e7c5a1ca-d462-4569-9932-d02769533478/volumes" Jul 11 07:48:43.363935 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:43.363901 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" event={"ID":"3e511a88-69fe-46a4-ab7b-f0aaa15c7545","Type":"ContainerStarted","Data":"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef"} Jul 11 07:48:43.364369 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:43.364046 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:48:43.383902 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:43.383853 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" podStartSLOduration=2.383838675 podStartE2EDuration="2.383838675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:48:43.382930317 +0000 UTC m=+613.228919209" watchObservedRunningTime="2026-07-11 07:48:43.383838675 +0000 UTC m=+613.229827566" Jul 11 07:48:54.372065 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:48:54.371994 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:49:11.657100 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.657062 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:49:11.657830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.657806 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7c5a1ca-d462-4569-9932-d02769533478" containerName="manager" Jul 11 07:49:11.657830 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.657828 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c5a1ca-d462-4569-9932-d02769533478" containerName="manager" Jul 11 07:49:11.657970 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.657892 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7c5a1ca-d462-4569-9932-d02769533478" containerName="manager" Jul 11 07:49:11.661029 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.661012 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.663292 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.663271 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 11 07:49:11.663402 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.663344 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6pgs\"" Jul 11 07:49:11.667540 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.667518 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:49:11.679207 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.679144 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:49:11.825826 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.825796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cntf8\" (UniqueName: \"kubernetes.io/projected/5cdac355-fa73-4646-82be-bffaf49a8633-kube-api-access-cntf8\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.826002 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.825866 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5cdac355-fa73-4646-82be-bffaf49a8633-config-file\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.926737 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.926634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5cdac355-fa73-4646-82be-bffaf49a8633-config-file\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.926737 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.926695 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cntf8\" (UniqueName: \"kubernetes.io/projected/5cdac355-fa73-4646-82be-bffaf49a8633-kube-api-access-cntf8\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.927339 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.927321 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5cdac355-fa73-4646-82be-bffaf49a8633-config-file\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.935718 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.935689 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cntf8\" (UniqueName: \"kubernetes.io/projected/5cdac355-fa73-4646-82be-bffaf49a8633-kube-api-access-cntf8\") pod \"limitador-limitador-54d6d988d5-4qtzf\" (UID: \"5cdac355-fa73-4646-82be-bffaf49a8633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:11.973415 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:11.973380 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:12.104760 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.104731 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:49:12.106663 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:49:12.106634 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cdac355_fa73_4646_82be_bffaf49a8633.slice/crio-6c6ef5ce7c1a9286da4945b9194f36762c444d761e85988f9991ac970957d5e2 WatchSource:0}: Error finding container 6c6ef5ce7c1a9286da4945b9194f36762c444d761e85988f9991ac970957d5e2: Status 404 returned error can't find the container with id 6c6ef5ce7c1a9286da4945b9194f36762c444d761e85988f9991ac970957d5e2 Jul 11 07:49:12.108485 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.108466 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 07:49:12.301996 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.301955 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:12.306470 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.306448 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:12.309134 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.309113 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h5kjn\"" Jul 11 07:49:12.311628 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.311606 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:12.431439 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.431406 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh2mg\" (UniqueName: \"kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg\") pod \"authorino-75d556bd77-qh7f2\" (UID: \"12f015d5-1e13-4ad3-8893-a3d86df078d9\") " pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:12.521132 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.521093 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" event={"ID":"5cdac355-fa73-4646-82be-bffaf49a8633","Type":"ContainerStarted","Data":"6c6ef5ce7c1a9286da4945b9194f36762c444d761e85988f9991ac970957d5e2"} Jul 11 07:49:12.532537 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.532507 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh2mg\" (UniqueName: \"kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg\") pod \"authorino-75d556bd77-qh7f2\" (UID: \"12f015d5-1e13-4ad3-8893-a3d86df078d9\") " pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:12.540606 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.540573 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh2mg\" (UniqueName: \"kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg\") pod \"authorino-75d556bd77-qh7f2\" (UID: \"12f015d5-1e13-4ad3-8893-a3d86df078d9\") " pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:12.617518 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.617425 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:12.785744 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:12.785689 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:12.788362 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:49:12.788319 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12f015d5_1e13_4ad3_8893_a3d86df078d9.slice/crio-2d09298cb4f328abcb0fd40f6214d1cfae1704c24aff173ef987a11ef7ad12d8 WatchSource:0}: Error finding container 2d09298cb4f328abcb0fd40f6214d1cfae1704c24aff173ef987a11ef7ad12d8: Status 404 returned error can't find the container with id 2d09298cb4f328abcb0fd40f6214d1cfae1704c24aff173ef987a11ef7ad12d8 Jul 11 07:49:13.532854 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:13.532121 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qh7f2" event={"ID":"12f015d5-1e13-4ad3-8893-a3d86df078d9","Type":"ContainerStarted","Data":"2d09298cb4f328abcb0fd40f6214d1cfae1704c24aff173ef987a11ef7ad12d8"} Jul 11 07:49:17.558459 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:17.558419 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" event={"ID":"5cdac355-fa73-4646-82be-bffaf49a8633","Type":"ContainerStarted","Data":"49b831b2a5b166afab596f446707ae6a130dcdfea24dc8a525508f983d545dda"} Jul 11 07:49:17.558966 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:17.558566 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:17.559909 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:17.559887 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qh7f2" event={"ID":"12f015d5-1e13-4ad3-8893-a3d86df078d9","Type":"ContainerStarted","Data":"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098"} Jul 11 07:49:17.574469 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:17.574425 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" podStartSLOduration=2.208700953 podStartE2EDuration="6.574411038s" totalImagesPullingTime="4.365710085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:49:17.572387217 +0000 UTC m=+647.418376109" watchObservedRunningTime="2026-07-11 07:49:17.574411038 +0000 UTC m=+647.420399930" Jul 11 07:49:17.586315 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:17.586272 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qh7f2" podStartSLOduration=1.901243802 podStartE2EDuration="5.586259909s" totalImagesPullingTime="3.685016107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:49:17.584906195 +0000 UTC m=+647.430895088" watchObservedRunningTime="2026-07-11 07:49:17.586259909 +0000 UTC m=+647.432248802" Jul 11 07:49:28.566970 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:28.566932 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4qtzf" Jul 11 07:49:43.482605 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.482560 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lz795"] Jul 11 07:49:43.492600 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.492567 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lz795"] Jul 11 07:49:43.492774 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.492686 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:43.628115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.628079 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxkxs\" (UniqueName: \"kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs\") pod \"authorino-5d4b67c446-lz795\" (UID: \"08178c9d-bd0c-4c13-b1d3-87bf874d379e\") " pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:43.700442 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.700409 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lz795"] Jul 11 07:49:43.700736 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:49:43.700715 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sxkxs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-lz795" podUID="08178c9d-bd0c-4c13-b1d3-87bf874d379e" Jul 11 07:49:43.719854 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.719816 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bfdbf4787-xbvbf"] Jul 11 07:49:43.722255 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.722239 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:43.728861 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.728835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxkxs\" (UniqueName: \"kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs\") pod \"authorino-5d4b67c446-lz795\" (UID: \"08178c9d-bd0c-4c13-b1d3-87bf874d379e\") " pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:43.730153 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.730134 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bfdbf4787-xbvbf"] Jul 11 07:49:43.741834 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.741808 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxkxs\" (UniqueName: \"kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs\") pod \"authorino-5d4b67c446-lz795\" (UID: \"08178c9d-bd0c-4c13-b1d3-87bf874d379e\") " pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:43.830019 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.829968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2jxk\" (UniqueName: \"kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk\") pod \"authorino-bfdbf4787-xbvbf\" (UID: \"815c1316-74b0-4b1c-ae28-9ed48c92a412\") " pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:43.878643 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.878609 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bfdbf4787-xbvbf"] Jul 11 07:49:43.878971 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:49:43.878948 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-n2jxk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-bfdbf4787-xbvbf" podUID="815c1316-74b0-4b1c-ae28-9ed48c92a412" Jul 11 07:49:43.904507 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.904461 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:49:43.907341 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.907312 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:43.910572 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.910545 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 11 07:49:43.917864 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.917831 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:49:43.930923 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.930883 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jxk\" (UniqueName: \"kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk\") pod \"authorino-bfdbf4787-xbvbf\" (UID: \"815c1316-74b0-4b1c-ae28-9ed48c92a412\") " pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:43.940092 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:43.940058 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2jxk\" (UniqueName: \"kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk\") pod \"authorino-bfdbf4787-xbvbf\" (UID: \"815c1316-74b0-4b1c-ae28-9ed48c92a412\") " pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:44.032409 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.032322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2szm6\" (UniqueName: \"kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.032409 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.032369 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.133079 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.133041 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2szm6\" (UniqueName: \"kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.133266 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.133092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.135489 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.135468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.140786 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.140762 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2szm6\" (UniqueName: \"kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6\") pod \"authorino-5696d55555-fjv8w\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.219091 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.219054 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:49:44.356391 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.356359 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:49:44.358111 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:49:44.358085 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a447caa_f3fc_41b7_b9c2_eaaf47a22cd2.slice/crio-52162ed5b8a48dd7b664b86b1d321622f4e956731a57af7fff094f32059ff92e WatchSource:0}: Error finding container 52162ed5b8a48dd7b664b86b1d321622f4e956731a57af7fff094f32059ff92e: Status 404 returned error can't find the container with id 52162ed5b8a48dd7b664b86b1d321622f4e956731a57af7fff094f32059ff92e Jul 11 07:49:44.705949 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.705926 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:44.706249 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.705926 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:44.710363 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.710338 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5696d55555-fjv8w" event={"ID":"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2","Type":"ContainerStarted","Data":"52162ed5b8a48dd7b664b86b1d321622f4e956731a57af7fff094f32059ff92e"} Jul 11 07:49:44.717848 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.717830 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:44.723296 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.723274 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:44.838598 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.838569 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxkxs\" (UniqueName: \"kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs\") pod \"08178c9d-bd0c-4c13-b1d3-87bf874d379e\" (UID: \"08178c9d-bd0c-4c13-b1d3-87bf874d379e\") " Jul 11 07:49:44.838789 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.838691 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2jxk\" (UniqueName: \"kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk\") pod \"815c1316-74b0-4b1c-ae28-9ed48c92a412\" (UID: \"815c1316-74b0-4b1c-ae28-9ed48c92a412\") " Jul 11 07:49:44.840756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.840720 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk" (OuterVolumeSpecName: "kube-api-access-n2jxk") pod "815c1316-74b0-4b1c-ae28-9ed48c92a412" (UID: "815c1316-74b0-4b1c-ae28-9ed48c92a412"). InnerVolumeSpecName "kube-api-access-n2jxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:49:44.840756 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.840725 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs" (OuterVolumeSpecName: "kube-api-access-sxkxs") pod "08178c9d-bd0c-4c13-b1d3-87bf874d379e" (UID: "08178c9d-bd0c-4c13-b1d3-87bf874d379e"). InnerVolumeSpecName "kube-api-access-sxkxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:49:44.940331 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.940239 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2jxk\" (UniqueName: \"kubernetes.io/projected/815c1316-74b0-4b1c-ae28-9ed48c92a412-kube-api-access-n2jxk\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:49:44.940331 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:44.940273 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxkxs\" (UniqueName: \"kubernetes.io/projected/08178c9d-bd0c-4c13-b1d3-87bf874d379e-kube-api-access-sxkxs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:49:45.712993 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.712952 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5696d55555-fjv8w" event={"ID":"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2","Type":"ContainerStarted","Data":"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3"} Jul 11 07:49:45.713411 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.713118 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfdbf4787-xbvbf" Jul 11 07:49:45.713411 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.713142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lz795" Jul 11 07:49:45.727598 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.727552 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5696d55555-fjv8w" podStartSLOduration=2.410783969 podStartE2EDuration="2.727535395s" totalImagesPullingTime="316.751426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:49:45.727119046 +0000 UTC m=+675.573107939" watchObservedRunningTime="2026-07-11 07:49:45.727535395 +0000 UTC m=+675.573524288" Jul 11 07:49:45.770597 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.770555 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bfdbf4787-xbvbf"] Jul 11 07:49:45.785117 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.785083 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:45.785304 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.785263 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qh7f2" podUID="12f015d5-1e13-4ad3-8893-a3d86df078d9" containerName="authorino" containerID="cri-o://f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098" gracePeriod=30 Jul 11 07:49:45.790776 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.790741 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bfdbf4787-xbvbf"] Jul 11 07:49:45.851667 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.851618 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lz795"] Jul 11 07:49:45.856276 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:45.856247 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lz795"] Jul 11 07:49:46.041157 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.041132 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:46.150532 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.150502 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh2mg\" (UniqueName: \"kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg\") pod \"12f015d5-1e13-4ad3-8893-a3d86df078d9\" (UID: \"12f015d5-1e13-4ad3-8893-a3d86df078d9\") " Jul 11 07:49:46.152665 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.152636 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg" (OuterVolumeSpecName: "kube-api-access-bh2mg") pod "12f015d5-1e13-4ad3-8893-a3d86df078d9" (UID: "12f015d5-1e13-4ad3-8893-a3d86df078d9"). InnerVolumeSpecName "kube-api-access-bh2mg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:49:46.251449 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.251414 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh2mg\" (UniqueName: \"kubernetes.io/projected/12f015d5-1e13-4ad3-8893-a3d86df078d9-kube-api-access-bh2mg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:49:46.711408 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.711377 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08178c9d-bd0c-4c13-b1d3-87bf874d379e" path="/var/lib/kubelet/pods/08178c9d-bd0c-4c13-b1d3-87bf874d379e/volumes" Jul 11 07:49:46.711651 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.711637 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="815c1316-74b0-4b1c-ae28-9ed48c92a412" path="/var/lib/kubelet/pods/815c1316-74b0-4b1c-ae28-9ed48c92a412/volumes" Jul 11 07:49:46.719370 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.719337 2572 generic.go:356] "Generic (PLEG): container finished" podID="12f015d5-1e13-4ad3-8893-a3d86df078d9" containerID="f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098" exitCode=0 Jul 11 07:49:46.719753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.719396 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qh7f2" Jul 11 07:49:46.719753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.719420 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qh7f2" event={"ID":"12f015d5-1e13-4ad3-8893-a3d86df078d9","Type":"ContainerDied","Data":"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098"} Jul 11 07:49:46.719753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.719474 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qh7f2" event={"ID":"12f015d5-1e13-4ad3-8893-a3d86df078d9","Type":"ContainerDied","Data":"2d09298cb4f328abcb0fd40f6214d1cfae1704c24aff173ef987a11ef7ad12d8"} Jul 11 07:49:46.719753 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.719500 2572 scope.go:117] "RemoveContainer" containerID="f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098" Jul 11 07:49:46.728425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.728408 2572 scope.go:117] "RemoveContainer" containerID="f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098" Jul 11 07:49:46.728742 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:49:46.728724 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098\": container with ID starting with f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098 not found: ID does not exist" containerID="f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098" Jul 11 07:49:46.728781 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.728760 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098"} err="failed to get container status \"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098\": rpc error: code = NotFound desc = could not find container \"f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098\": container with ID starting with f976e8cd878bdc2f10c34e648cbe5905ad57e16b39cfd616b01d70cd55973098 not found: ID does not exist" Jul 11 07:49:46.735481 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.735455 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:46.742275 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:46.742238 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qh7f2"] Jul 11 07:49:48.711852 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:49:48.711819 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f015d5-1e13-4ad3-8893-a3d86df078d9" path="/var/lib/kubelet/pods/12f015d5-1e13-4ad3-8893-a3d86df078d9/volumes" Jul 11 07:50:26.432876 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:50:26.432840 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:50:41.295845 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:50:41.295805 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:50:55.177438 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:50:55.177401 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:51:12.178940 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:51:12.178906 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:51:29.288441 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:51:29.288401 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:51:42.180624 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:51:42.180584 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:51:52.579052 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:51:52.578966 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:53:10.581500 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.581461 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7568c9f894-wvbfg"] Jul 11 07:53:10.582147 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.582128 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12f015d5-1e13-4ad3-8893-a3d86df078d9" containerName="authorino" Jul 11 07:53:10.582195 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.582151 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f015d5-1e13-4ad3-8893-a3d86df078d9" containerName="authorino" Jul 11 07:53:10.582264 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.582253 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="12f015d5-1e13-4ad3-8893-a3d86df078d9" containerName="authorino" Jul 11 07:53:10.585903 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.585884 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.590174 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.590148 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7568c9f894-wvbfg"] Jul 11 07:53:10.640156 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.640110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjhmh\" (UniqueName: \"kubernetes.io/projected/d86947d1-1b27-404c-97e6-043aae55dc77-kube-api-access-fjhmh\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.640347 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.640180 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d86947d1-1b27-404c-97e6-043aae55dc77-tls-cert\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.741764 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.741692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjhmh\" (UniqueName: \"kubernetes.io/projected/d86947d1-1b27-404c-97e6-043aae55dc77-kube-api-access-fjhmh\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.741974 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.741924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d86947d1-1b27-404c-97e6-043aae55dc77-tls-cert\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.744485 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.744460 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d86947d1-1b27-404c-97e6-043aae55dc77-tls-cert\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.749425 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.749400 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjhmh\" (UniqueName: \"kubernetes.io/projected/d86947d1-1b27-404c-97e6-043aae55dc77-kube-api-access-fjhmh\") pod \"authorino-7568c9f894-wvbfg\" (UID: \"d86947d1-1b27-404c-97e6-043aae55dc77\") " pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:10.897713 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:10.897607 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568c9f894-wvbfg" Jul 11 07:53:11.032922 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.032893 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7568c9f894-wvbfg"] Jul 11 07:53:11.034880 ip-10-0-132-10 kubenswrapper[2572]: W0711 07:53:11.034851 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd86947d1_1b27_404c_97e6_043aae55dc77.slice/crio-e766f3eb9b46f74a1bf0cdb1ca10833ed9a7f386d7fe9464ce2c4f54901a047e WatchSource:0}: Error finding container e766f3eb9b46f74a1bf0cdb1ca10833ed9a7f386d7fe9464ce2c4f54901a047e: Status 404 returned error can't find the container with id e766f3eb9b46f74a1bf0cdb1ca10833ed9a7f386d7fe9464ce2c4f54901a047e Jul 11 07:53:11.829388 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.829351 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568c9f894-wvbfg" event={"ID":"d86947d1-1b27-404c-97e6-043aae55dc77","Type":"ContainerStarted","Data":"bba719d255749add0b9fd8b5d6c7d8ec78303490faa377e79427211c783670a2"} Jul 11 07:53:11.830063 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.830036 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568c9f894-wvbfg" event={"ID":"d86947d1-1b27-404c-97e6-043aae55dc77","Type":"ContainerStarted","Data":"e766f3eb9b46f74a1bf0cdb1ca10833ed9a7f386d7fe9464ce2c4f54901a047e"} Jul 11 07:53:11.848376 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.848301 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7568c9f894-wvbfg" podStartSLOduration=1.480387888 podStartE2EDuration="1.84828217s" totalImagesPullingTime="367.894282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 07:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 07:53:11.847719843 +0000 UTC m=+881.693708728" watchObservedRunningTime="2026-07-11 07:53:11.84828217 +0000 UTC m=+881.694271064" Jul 11 07:53:11.886184 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.886131 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:53:11.886618 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:11.886577 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5696d55555-fjv8w" podUID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" containerName="authorino" containerID="cri-o://83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3" gracePeriod=30 Jul 11 07:53:12.145546 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.145520 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:53:12.255989 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.255953 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2szm6\" (UniqueName: \"kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6\") pod \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " Jul 11 07:53:12.256174 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.256059 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert\") pod \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\" (UID: \"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2\") " Jul 11 07:53:12.258075 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.258039 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6" (OuterVolumeSpecName: "kube-api-access-2szm6") pod "6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" (UID: "6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2"). InnerVolumeSpecName "kube-api-access-2szm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:53:12.267066 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.267036 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" (UID: "6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 07:53:12.357560 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.357523 2572 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-tls-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:53:12.357560 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.357555 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2szm6\" (UniqueName: \"kubernetes.io/projected/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2-kube-api-access-2szm6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:53:12.838468 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.838425 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" containerID="83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3" exitCode=0 Jul 11 07:53:12.838961 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.838488 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5696d55555-fjv8w" Jul 11 07:53:12.838961 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.838524 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5696d55555-fjv8w" event={"ID":"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2","Type":"ContainerDied","Data":"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3"} Jul 11 07:53:12.838961 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.838576 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5696d55555-fjv8w" event={"ID":"6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2","Type":"ContainerDied","Data":"52162ed5b8a48dd7b664b86b1d321622f4e956731a57af7fff094f32059ff92e"} Jul 11 07:53:12.838961 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.838607 2572 scope.go:117] "RemoveContainer" containerID="83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3" Jul 11 07:53:12.847692 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.847666 2572 scope.go:117] "RemoveContainer" containerID="83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3" Jul 11 07:53:12.848156 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:53:12.848128 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3\": container with ID starting with 83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3 not found: ID does not exist" containerID="83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3" Jul 11 07:53:12.848267 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.848165 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3"} err="failed to get container status \"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3\": rpc error: code = NotFound desc = could not find container \"83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3\": container with ID starting with 83836e85640bcbe861553a071824f487565f29e2a7c967d53ccca7907fed52b3 not found: ID does not exist" Jul 11 07:53:12.859227 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.859201 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:53:12.862742 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:12.862720 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5696d55555-fjv8w"] Jul 11 07:53:14.712205 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:14.712160 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" path="/var/lib/kubelet/pods/6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2/volumes" Jul 11 07:53:30.606074 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:30.606048 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:53:30.606833 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:30.606813 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:53:41.478165 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:41.478124 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:53:50.877791 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:53:50.877748 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:54:00.577131 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:54:00.577090 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:54:10.883685 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:54:10.883642 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:54:20.781980 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:54:20.781927 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:54:47.678542 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:54:47.678499 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:55:00.590686 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:55:00.590603 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:55:31.779731 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:55:31.779673 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:55:41.086151 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:55:41.086120 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:55:50.279340 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:55:50.279305 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:56:01.281454 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:56:01.281414 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:56:12.583401 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:56:12.583363 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:56:22.480373 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:56:22.480276 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:56:33.279768 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:56:33.279730 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:56:43.185552 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:56:43.185517 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:57:05.478816 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:57:05.478781 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:57:15.985185 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:57:15.985147 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:57:38.079619 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:57:38.079580 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:57:49.085580 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:57:49.085542 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:58:03.674590 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:03.674493 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:58:12.887530 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:12.887498 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:58:24.677765 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:24.677725 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:58:30.650115 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:30.650084 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:58:30.652114 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:30.652092 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 07:58:33.580265 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:33.580230 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:58:42.877967 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:58:42.877935 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 07:59:08.170079 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:08.170047 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:59:08.170616 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:08.170251 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" podUID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" containerName="manager" containerID="cri-o://6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef" gracePeriod=10 Jul 11 07:59:11.814913 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.814889 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:59:11.824155 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.824130 2572 generic.go:356] "Generic (PLEG): container finished" podID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" containerID="6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef" exitCode=0 Jul 11 07:59:11.824265 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.824165 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" Jul 11 07:59:11.824265 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.824217 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" event={"ID":"3e511a88-69fe-46a4-ab7b-f0aaa15c7545","Type":"ContainerDied","Data":"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef"} Jul 11 07:59:11.824393 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.824293 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm" event={"ID":"3e511a88-69fe-46a4-ab7b-f0aaa15c7545","Type":"ContainerDied","Data":"41e18e71cfd2d21da4f17140ffc502c8c54d96b888a4111f52affcbc7449c708"} Jul 11 07:59:11.824393 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.824328 2572 scope.go:117] "RemoveContainer" containerID="6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef" Jul 11 07:59:11.833986 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.833718 2572 scope.go:117] "RemoveContainer" containerID="6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef" Jul 11 07:59:11.834214 ip-10-0-132-10 kubenswrapper[2572]: E0711 07:59:11.834187 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef\": container with ID starting with 6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef not found: ID does not exist" containerID="6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef" Jul 11 07:59:11.834299 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.834226 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef"} err="failed to get container status \"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef\": rpc error: code = NotFound desc = could not find container \"6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef\": container with ID starting with 6fd3479ad4ce72c77de120f620824d2753952f7808fb42f17fba297707fe2aef not found: ID does not exist" Jul 11 07:59:11.853518 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.853485 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-754w5\" (UniqueName: \"kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5\") pod \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " Jul 11 07:59:11.853653 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.853555 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume\") pod \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\" (UID: \"3e511a88-69fe-46a4-ab7b-f0aaa15c7545\") " Jul 11 07:59:11.854007 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.853981 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3e511a88-69fe-46a4-ab7b-f0aaa15c7545" (UID: "3e511a88-69fe-46a4-ab7b-f0aaa15c7545"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 07:59:11.855900 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.855873 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5" (OuterVolumeSpecName: "kube-api-access-754w5") pod "3e511a88-69fe-46a4-ab7b-f0aaa15c7545" (UID: "3e511a88-69fe-46a4-ab7b-f0aaa15c7545"). InnerVolumeSpecName "kube-api-access-754w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 07:59:11.954629 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.954557 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-754w5\" (UniqueName: \"kubernetes.io/projected/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-kube-api-access-754w5\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:59:11.954629 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:11.954583 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e511a88-69fe-46a4-ab7b-f0aaa15c7545-extensions-socket-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 07:59:12.146459 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:12.146423 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:59:12.149826 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:12.149800 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p5lqm"] Jul 11 07:59:12.710597 ip-10-0-132-10 kubenswrapper[2572]: I0711 07:59:12.710566 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" path="/var/lib/kubelet/pods/3e511a88-69fe-46a4-ab7b-f0aaa15c7545/volumes" Jul 11 08:00:00.143192 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143156 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn"] Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143545 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" containerName="authorino" Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143555 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" containerName="authorino" Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143569 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" containerName="manager" Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143574 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" containerName="manager" Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143626 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e511a88-69fe-46a4-ab7b-f0aaa15c7545" containerName="manager" Jul 11 08:00:00.143685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.143633 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a447caa-f3fc-41b7-b9c2-eaaf47a22cd2" containerName="authorino" Jul 11 08:00:00.146514 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.146499 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" Jul 11 08:00:00.149103 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.149084 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:00:00.154398 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.153875 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn"] Jul 11 08:00:00.157610 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.157587 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" Jul 11 08:00:00.284246 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.284217 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn"] Jul 11 08:00:00.286466 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:00:00.286438 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2df565_b9e5_4bfb_bf43_14b33829abae.slice/crio-90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961 WatchSource:0}: Error finding container 90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961: Status 404 returned error can't find the container with id 90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961 Jul 11 08:00:00.288282 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:00.288262 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 08:00:01.092985 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:01.092953 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" event={"ID":"3a2df565-b9e5-4bfb-bf43-14b33829abae","Type":"ContainerStarted","Data":"90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961"} Jul 11 08:00:05.122176 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:05.122074 2572 generic.go:356] "Generic (PLEG): container finished" podID="3a2df565-b9e5-4bfb-bf43-14b33829abae" containerID="5c5907f691a13b4d45c6ac65da6a06dd315a1d9b8b1d4ba5f0f939dd12e2d7f4" exitCode=0 Jul 11 08:00:05.122176 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:05.122161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" event={"ID":"3a2df565-b9e5-4bfb-bf43-14b33829abae","Type":"ContainerDied","Data":"5c5907f691a13b4d45c6ac65da6a06dd315a1d9b8b1d4ba5f0f939dd12e2d7f4"} Jul 11 08:00:06.263469 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:06.263447 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" Jul 11 08:00:07.134557 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:07.134527 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" Jul 11 08:00:07.134748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:07.134527 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729280-7kfjn" event={"ID":"3a2df565-b9e5-4bfb-bf43-14b33829abae","Type":"ContainerDied","Data":"90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961"} Jul 11 08:00:07.134748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:07.134648 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90631d5d30e96951665fb5477cc8b87efabc47ccfd563a23ff684ac218abe961" Jul 11 08:00:09.288050 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.288011 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv"] Jul 11 08:00:09.288628 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.288606 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a2df565-b9e5-4bfb-bf43-14b33829abae" containerName="cleanup" Jul 11 08:00:09.288753 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.288630 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a2df565-b9e5-4bfb-bf43-14b33829abae" containerName="cleanup" Jul 11 08:00:09.288753 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.288743 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a2df565-b9e5-4bfb-bf43-14b33829abae" containerName="cleanup" Jul 11 08:00:09.291908 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.291886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.294495 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.294477 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f5qqf\"" Jul 11 08:00:09.303116 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.303094 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv"] Jul 11 08:00:09.330674 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.330643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ls92\" (UniqueName: \"kubernetes.io/projected/55425168-df33-4415-a2ef-53234dbc2b97-kube-api-access-9ls92\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.330851 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.330686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55425168-df33-4415-a2ef-53234dbc2b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.431724 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.431675 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ls92\" (UniqueName: \"kubernetes.io/projected/55425168-df33-4415-a2ef-53234dbc2b97-kube-api-access-9ls92\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.431868 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.431745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55425168-df33-4415-a2ef-53234dbc2b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.432164 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.432146 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55425168-df33-4415-a2ef-53234dbc2b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.439489 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.439466 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ls92\" (UniqueName: \"kubernetes.io/projected/55425168-df33-4415-a2ef-53234dbc2b97-kube-api-access-9ls92\") pod \"kuadrant-operator-controller-manager-86878c769f-mbbrv\" (UID: \"55425168-df33-4415-a2ef-53234dbc2b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.604795 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.604694 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:09.742454 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:09.742430 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv"] Jul 11 08:00:09.744421 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:00:09.744391 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55425168_df33_4415_a2ef_53234dbc2b97.slice/crio-4d69ee9114a3652e78b9001e5bf6f978bbafc35c35ca4fd2fc30fb85e9d257b7 WatchSource:0}: Error finding container 4d69ee9114a3652e78b9001e5bf6f978bbafc35c35ca4fd2fc30fb85e9d257b7: Status 404 returned error can't find the container with id 4d69ee9114a3652e78b9001e5bf6f978bbafc35c35ca4fd2fc30fb85e9d257b7 Jul 11 08:00:10.153859 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:10.153822 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" event={"ID":"55425168-df33-4415-a2ef-53234dbc2b97","Type":"ContainerStarted","Data":"24ecda1b40f871afe652cc6d9d015d33d0dcf92174eb0784b2d2ccaa6160aa42"} Jul 11 08:00:10.154055 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:10.153871 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" event={"ID":"55425168-df33-4415-a2ef-53234dbc2b97","Type":"ContainerStarted","Data":"4d69ee9114a3652e78b9001e5bf6f978bbafc35c35ca4fd2fc30fb85e9d257b7"} Jul 11 08:00:10.154055 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:10.153941 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:10.175585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:10.175532 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" podStartSLOduration=1.175517561 podStartE2EDuration="1.175517561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:00:10.17299131 +0000 UTC m=+1300.018980195" watchObservedRunningTime="2026-07-11 08:00:10.175517561 +0000 UTC m=+1300.021506512" Jul 11 08:00:21.161216 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:21.161187 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mbbrv" Jul 11 08:00:30.488255 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:30.488212 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:00:39.283272 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:39.283230 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:00:52.485982 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:00:52.485907 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:03.987591 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:03.987555 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:16.180173 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:16.180138 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:25.587765 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:25.587730 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:34.180389 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:34.180352 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:45.583269 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:45.583231 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:01:56.576605 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:01:56.576567 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:02:05.478007 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:02:05.477960 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:02:16.476678 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:02:16.476641 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:02:40.276189 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:02:40.276111 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:02:47.781876 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:02:47.781810 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:02:57.586750 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:02:57.586713 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:07.676388 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:07.676349 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:16.488906 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:16.488870 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:29.089445 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:29.089398 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:30.694451 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:30.694413 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:03:30.697903 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:30.697882 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:03:38.682652 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:38.682613 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:48.386934 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:48.386902 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:03:55.685212 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:03:55.685135 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:06.881800 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:06.881757 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:16.383445 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:16.383409 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:29.487294 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:29.487257 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:38.588811 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:38.588772 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:48.390320 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:48.390288 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:04:57.774677 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:04:57.774644 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:05:11.582552 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:05:11.582516 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:05:19.983779 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:05:19.983739 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:05:32.979060 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:05:32.978969 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:05:43.690607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:05:43.690571 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:05:53.983646 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:05:53.983605 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:06:00.577726 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:00.577671 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:06:12.786084 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.786049 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:06:12.789749 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.789696 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.792317 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.792297 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8bc07405-serving-cert\"" Jul 11 08:06:12.792462 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.792451 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jhxtl\"" Jul 11 08:06:12.792531 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.792463 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:06:12.793458 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.793444 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 11 08:06:12.799640 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.799618 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:06:12.855186 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.855156 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f87h\" (UniqueName: \"kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.855186 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.855196 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.855393 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.855325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.956074 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.956042 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f87h\" (UniqueName: \"kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.956074 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.956078 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.956335 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.956166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.956335 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:06:12.956269 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-serving-cert: secret "maas-api-e2e-ait-8bc07405-serving-cert" not found Jul 11 08:06:12.956441 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:06:12.956359 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls podName:68dbff1c-b017-4d65-9ec9-c49cac132138 nodeName:}" failed. No retries permitted until 2026-07-11 08:06:13.456335902 +0000 UTC m=+1663.302324773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls") pod "maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" (UID: "68dbff1c-b017-4d65-9ec9-c49cac132138") : secret "maas-api-e2e-ait-8bc07405-serving-cert" not found Jul 11 08:06:12.956819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.956801 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:12.966640 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:12.966617 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f87h\" (UniqueName: \"kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:13.460828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:13.460783 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:13.463176 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:13.463153 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") pod \"maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:13.702803 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:13.702773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:13.840290 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:13.840264 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:06:13.842308 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:06:13.842280 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dbff1c_b017_4d65_9ec9_c49cac132138.slice/crio-49c4dbb9122f94a474d1ac3b481236ca4d57e01c74f8186b4fad7e4fc7f9a670 WatchSource:0}: Error finding container 49c4dbb9122f94a474d1ac3b481236ca4d57e01c74f8186b4fad7e4fc7f9a670: Status 404 returned error can't find the container with id 49c4dbb9122f94a474d1ac3b481236ca4d57e01c74f8186b4fad7e4fc7f9a670 Jul 11 08:06:13.843603 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:13.843586 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 08:06:14.154015 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:14.153984 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" event={"ID":"68dbff1c-b017-4d65-9ec9-c49cac132138","Type":"ContainerStarted","Data":"49c4dbb9122f94a474d1ac3b481236ca4d57e01c74f8186b4fad7e4fc7f9a670"} Jul 11 08:06:15.787997 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:15.787916 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:06:16.166194 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:16.166111 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" event={"ID":"68dbff1c-b017-4d65-9ec9-c49cac132138","Type":"ContainerStarted","Data":"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65"} Jul 11 08:06:16.166194 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:16.166174 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:16.181838 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:16.181786 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" podStartSLOduration=2.574568178 podStartE2EDuration="4.181772295s" totalImagesPullingTime="1.607204117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:06:16.181073249 +0000 UTC m=+1666.027062158" watchObservedRunningTime="2026-07-11 08:06:16.181772295 +0000 UTC m=+1666.027761187" Jul 11 08:06:17.778667 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:17.778630 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:17.782518 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:17.782501 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:17.785561 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:17.785528 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:06:17.788502 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:17.788476 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:17.904780 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:17.904747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q\" (UID: \"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:18.005601 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:18.005567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q\" (UID: \"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:18.006197 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:18.006177 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q\" (UID: \"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:18.094484 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:18.094401 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:18.224277 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:18.224248 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:18.225656 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:06:18.225627 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fafe61e_5bc9_4b51_ae85_6a6f636c3e34.slice/crio-1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941 WatchSource:0}: Error finding container 1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941: Status 404 returned error can't find the container with id 1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941 Jul 11 08:06:19.190432 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:19.190342 2572 generic.go:356] "Generic (PLEG): container finished" podID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerID="16fa17ec89e4deac9fa398cba7cafe78d48ca3c18707a38f2383edcfe6f4399f" exitCode=7 Jul 11 08:06:19.190432 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:19.190403 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerDied","Data":"16fa17ec89e4deac9fa398cba7cafe78d48ca3c18707a38f2383edcfe6f4399f"} Jul 11 08:06:19.190958 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:19.190483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerStarted","Data":"1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941"} Jul 11 08:06:19.190958 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:19.190688 2572 scope.go:117] "RemoveContainer" containerID="16fa17ec89e4deac9fa398cba7cafe78d48ca3c18707a38f2383edcfe6f4399f" Jul 11 08:06:20.198009 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:20.197912 2572 generic.go:356] "Generic (PLEG): container finished" podID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerID="7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97" exitCode=7 Jul 11 08:06:20.198009 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:20.197992 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerDied","Data":"7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97"} Jul 11 08:06:20.198508 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:20.198067 2572 scope.go:117] "RemoveContainer" containerID="16fa17ec89e4deac9fa398cba7cafe78d48ca3c18707a38f2383edcfe6f4399f" Jul 11 08:06:20.198508 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:20.198320 2572 scope.go:117] "RemoveContainer" containerID="7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97" Jul 11 08:06:20.198603 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:06:20.198558 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q_odh-ai-gateway-infra(7fafe61e-5bc9-4b51-ae85-6a6f636c3e34)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" Jul 11 08:06:21.205648 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:21.205624 2572 scope.go:117] "RemoveContainer" containerID="7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97" Jul 11 08:06:21.206125 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:06:21.205915 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q_odh-ai-gateway-infra(7fafe61e-5bc9-4b51-ae85-6a6f636c3e34)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" Jul 11 08:06:22.177887 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:22.177857 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:06:33.705793 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:33.705762 2572 scope.go:117] "RemoveContainer" containerID="7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97" Jul 11 08:06:34.285478 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:34.285443 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerStarted","Data":"d519e0f2bd9fa1261a199d651bb6f1539871e47b0cfa518cf0ba7922daf64a20"} Jul 11 08:06:34.301425 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:34.301372 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" podStartSLOduration=17.301357070999998 podStartE2EDuration="17.301357071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:06:34.299691594 +0000 UTC m=+1684.145680502" watchObservedRunningTime="2026-07-11 08:06:34.301357071 +0000 UTC m=+1684.147345957" Jul 11 08:06:35.299072 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:35.299038 2572 generic.go:356] "Generic (PLEG): container finished" podID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerID="d519e0f2bd9fa1261a199d651bb6f1539871e47b0cfa518cf0ba7922daf64a20" exitCode=0 Jul 11 08:06:35.299525 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:35.299113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerDied","Data":"d519e0f2bd9fa1261a199d651bb6f1539871e47b0cfa518cf0ba7922daf64a20"} Jul 11 08:06:35.299525 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:35.299194 2572 scope.go:117] "RemoveContainer" containerID="7f48e9834b37fe722e9ab99dc697f637ec40f09ce455e2c0702858aefedaee97" Jul 11 08:06:35.310732 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:35.310691 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:36.437129 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:36.437105 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:36.566381 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:36.566303 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca\") pod \"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34\" (UID: \"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34\") " Jul 11 08:06:36.566628 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:36.566607 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" (UID: "7fafe61e-5bc9-4b51-ae85-6a6f636c3e34"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:06:36.667586 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:36.667557 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:06:37.313017 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:37.312989 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" Jul 11 08:06:37.313219 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:37.313024 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q" event={"ID":"7fafe61e-5bc9-4b51-ae85-6a6f636c3e34","Type":"ContainerDied","Data":"1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941"} Jul 11 08:06:37.313219 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:37.313070 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1303babe48bbad60195ae0cc61d85b032444e6617a573a541485cbd186b11941" Jul 11 08:06:37.344148 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:37.344118 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:37.346016 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:37.345998 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-k2k6q"] Jul 11 08:06:38.711393 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:06:38.711360 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" path="/var/lib/kubelet/pods/7fafe61e-5bc9-4b51-ae85-6a6f636c3e34/volumes" Jul 11 08:07:07.817588 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.817501 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh"] Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818060 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818078 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818104 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818113 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818125 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818134 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818481 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818230 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.818481 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.818243 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:07.821499 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.821478 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:07.824398 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.824368 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:07:07.827099 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.827077 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh"] Jul 11 08:07:07.950153 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:07.950119 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh\" (UID: \"ac012bab-c750-4e5c-89c6-a923fd124973\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:08.051151 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:08.051103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh\" (UID: \"ac012bab-c750-4e5c-89c6-a923fd124973\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:08.051739 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:08.051720 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh\" (UID: \"ac012bab-c750-4e5c-89c6-a923fd124973\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:08.133125 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:08.133053 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:08.260553 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:08.260524 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh"] Jul 11 08:07:08.260843 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:07:08.260812 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac012bab_c750_4e5c_89c6_a923fd124973.slice/crio-d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982 WatchSource:0}: Error finding container d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982: Status 404 returned error can't find the container with id d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982 Jul 11 08:07:08.482388 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:08.482361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" event={"ID":"ac012bab-c750-4e5c-89c6-a923fd124973","Type":"ContainerStarted","Data":"d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982"} Jul 11 08:07:09.495107 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:09.495071 2572 generic.go:356] "Generic (PLEG): container finished" podID="ac012bab-c750-4e5c-89c6-a923fd124973" containerID="7ae5b38b133c470987ec7cef03273cd34f97e7ce01ff607ea02c0b75d679ad08" exitCode=0 Jul 11 08:07:09.495479 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:09.495153 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" event={"ID":"ac012bab-c750-4e5c-89c6-a923fd124973","Type":"ContainerDied","Data":"7ae5b38b133c470987ec7cef03273cd34f97e7ce01ff607ea02c0b75d679ad08"} Jul 11 08:07:10.633609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:10.633585 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:10.776868 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:10.776796 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca\") pod \"ac012bab-c750-4e5c-89c6-a923fd124973\" (UID: \"ac012bab-c750-4e5c-89c6-a923fd124973\") " Jul 11 08:07:10.777118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:10.777096 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac012bab-c750-4e5c-89c6-a923fd124973" (UID: "ac012bab-c750-4e5c-89c6-a923fd124973"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:07:10.877634 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:10.877603 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac012bab-c750-4e5c-89c6-a923fd124973-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:07:11.507242 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:11.507213 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" Jul 11 08:07:11.507439 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:11.507243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh" event={"ID":"ac012bab-c750-4e5c-89c6-a923fd124973","Type":"ContainerDied","Data":"d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982"} Jul 11 08:07:11.507439 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:11.507285 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d228ae34dd9c828c60adb08b9fca0f6d91ddb841a3647c2685cfe82e03f97982" Jul 11 08:07:18.181980 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.181941 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:07:18.182478 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.182219 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" podUID="68dbff1c-b017-4d65-9ec9-c49cac132138" containerName="maas-api" containerID="cri-o://48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65" gracePeriod=30 Jul 11 08:07:18.429006 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.428982 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:07:18.544685 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.544655 2572 generic.go:356] "Generic (PLEG): container finished" podID="68dbff1c-b017-4d65-9ec9-c49cac132138" containerID="48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65" exitCode=0 Jul 11 08:07:18.544862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.544726 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" Jul 11 08:07:18.544862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.544744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" event={"ID":"68dbff1c-b017-4d65-9ec9-c49cac132138","Type":"ContainerDied","Data":"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65"} Jul 11 08:07:18.544862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.544801 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k" event={"ID":"68dbff1c-b017-4d65-9ec9-c49cac132138","Type":"ContainerDied","Data":"49c4dbb9122f94a474d1ac3b481236ca4d57e01c74f8186b4fad7e4fc7f9a670"} Jul 11 08:07:18.544862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.544832 2572 scope.go:117] "RemoveContainer" containerID="48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65" Jul 11 08:07:18.550809 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.550786 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca\") pod \"68dbff1c-b017-4d65-9ec9-c49cac132138\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " Jul 11 08:07:18.550942 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.550881 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f87h\" (UniqueName: \"kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h\") pod \"68dbff1c-b017-4d65-9ec9-c49cac132138\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " Jul 11 08:07:18.550942 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.550904 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") pod \"68dbff1c-b017-4d65-9ec9-c49cac132138\" (UID: \"68dbff1c-b017-4d65-9ec9-c49cac132138\") " Jul 11 08:07:18.551107 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.551086 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68dbff1c-b017-4d65-9ec9-c49cac132138" (UID: "68dbff1c-b017-4d65-9ec9-c49cac132138"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:07:18.552827 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.552806 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68dbff1c-b017-4d65-9ec9-c49cac132138" (UID: "68dbff1c-b017-4d65-9ec9-c49cac132138"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 08:07:18.553222 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.553206 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h" (OuterVolumeSpecName: "kube-api-access-6f87h") pod "68dbff1c-b017-4d65-9ec9-c49cac132138" (UID: "68dbff1c-b017-4d65-9ec9-c49cac132138"). InnerVolumeSpecName "kube-api-access-6f87h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:07:18.554198 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.554179 2572 scope.go:117] "RemoveContainer" containerID="48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65" Jul 11 08:07:18.554467 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:07:18.554443 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65\": container with ID starting with 48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65 not found: ID does not exist" containerID="48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65" Jul 11 08:07:18.554524 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.554472 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65"} err="failed to get container status \"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65\": rpc error: code = NotFound desc = could not find container \"48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65\": container with ID starting with 48c8145d66dcc36eb5063396a14c69ff4e830666698db955811ec0a7343bed65 not found: ID does not exist" Jul 11 08:07:18.652458 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.652417 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f87h\" (UniqueName: \"kubernetes.io/projected/68dbff1c-b017-4d65-9ec9-c49cac132138-kube-api-access-6f87h\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:07:18.652458 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.652454 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68dbff1c-b017-4d65-9ec9-c49cac132138-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:07:18.652458 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.652463 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68dbff1c-b017-4d65-9ec9-c49cac132138-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:07:18.860936 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.860851 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:07:18.866320 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:18.866295 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8bc07405-57688fcb9b-rcx7k"] Jul 11 08:07:20.711598 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:20.711569 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68dbff1c-b017-4d65-9ec9-c49cac132138" path="/var/lib/kubelet/pods/68dbff1c-b017-4d65-9ec9-c49cac132138/volumes" Jul 11 08:07:21.076217 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:21.076183 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:07:33.669819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.669770 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:07:33.670395 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670374 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dbff1c-b017-4d65-9ec9-c49cac132138" containerName="maas-api" Jul 11 08:07:33.670470 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670399 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dbff1c-b017-4d65-9ec9-c49cac132138" containerName="maas-api" Jul 11 08:07:33.670470 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670425 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac012bab-c750-4e5c-89c6-a923fd124973" containerName="revoke-keys" Jul 11 08:07:33.670470 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670434 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac012bab-c750-4e5c-89c6-a923fd124973" containerName="revoke-keys" Jul 11 08:07:33.670580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670534 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac012bab-c750-4e5c-89c6-a923fd124973" containerName="revoke-keys" Jul 11 08:07:33.670580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670548 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dbff1c-b017-4d65-9ec9-c49cac132138" containerName="maas-api" Jul 11 08:07:33.670580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.670559 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fafe61e-5bc9-4b51-ae85-6a6f636c3e34" containerName="revoke-keys" Jul 11 08:07:33.673908 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.673886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.676693 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.676668 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7f7e8a77-gw-openshift-default-dockercfg-pcwfw\"" Jul 11 08:07:33.680076 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680017 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680217 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680108 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680217 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5tg7\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680339 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680339 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680332 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680449 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680449 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680549 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680454 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.680549 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.680482 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.686866 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.686842 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:07:33.781444 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781407 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5tg7\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781526 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781553 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781624 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.781660 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781653 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782011 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782011 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.781932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782159 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.782139 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782227 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.782196 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782284 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.782253 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782343 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.782325 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.782539 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.782517 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.784367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.784339 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.784483 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.784414 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.790800 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.790773 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.790977 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.790956 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5tg7\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7\") pod \"e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:33.985345 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:33.985310 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:34.128198 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.128166 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:07:34.130033 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:07:34.130009 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf986d49e_ea75_454e_8e21_88e4a984cf5d.slice/crio-4e620783417891bae2117a95d0ce6c2882fc4e56dada74c8dd50b11ab734d929 WatchSource:0}: Error finding container 4e620783417891bae2117a95d0ce6c2882fc4e56dada74c8dd50b11ab734d929: Status 404 returned error can't find the container with id 4e620783417891bae2117a95d0ce6c2882fc4e56dada74c8dd50b11ab734d929 Jul 11 08:07:34.132312 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.132272 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:34.132381 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.132360 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:34.132420 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.132401 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:34.642092 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.642053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerStarted","Data":"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365"} Jul 11 08:07:34.642241 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.642122 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerStarted","Data":"4e620783417891bae2117a95d0ce6c2882fc4e56dada74c8dd50b11ab734d929"} Jul 11 08:07:34.661847 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.661785 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podStartSLOduration=1.661765107 podStartE2EDuration="1.661765107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:07:34.661537007 +0000 UTC m=+1744.507525902" watchObservedRunningTime="2026-07-11 08:07:34.661765107 +0000 UTC m=+1744.507754002" Jul 11 08:07:34.986454 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:34.986426 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:35.991213 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:35.991175 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:36.654260 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:36.654227 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:36.655276 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:36.655257 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:39.371541 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.371507 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:07:39.375206 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.375189 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:39.377605 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.377582 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:07:39.377769 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.377698 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:07:39.382290 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.382259 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:07:39.436517 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.436482 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5\" (UID: \"3b9e033a-7b0a-408b-8100-374fed544d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:39.537329 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.537260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5\" (UID: \"3b9e033a-7b0a-408b-8100-374fed544d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:39.538134 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.538108 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5\" (UID: \"3b9e033a-7b0a-408b-8100-374fed544d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:39.687657 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.687560 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:39.823187 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:39.823164 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:07:39.825214 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:07:39.825189 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9e033a_7b0a_408b_8100_374fed544d8f.slice/crio-9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525 WatchSource:0}: Error finding container 9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525: Status 404 returned error can't find the container with id 9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525 Jul 11 08:07:40.579023 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:07:40.578990 2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9e033a_7b0a_408b_8100_374fed544d8f.slice/crio-conmon-b8e756f8867e9f05dbd1ed2fbeeb7368a8f80ddd6c31c0bd8349050810ef8c16.scope\": RecentStats: unable to find data in memory cache]" Jul 11 08:07:40.677046 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:40.677013 2572 generic.go:356] "Generic (PLEG): container finished" podID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerID="b8e756f8867e9f05dbd1ed2fbeeb7368a8f80ddd6c31c0bd8349050810ef8c16" exitCode=7 Jul 11 08:07:40.677231 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:40.677096 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" event={"ID":"3b9e033a-7b0a-408b-8100-374fed544d8f","Type":"ContainerDied","Data":"b8e756f8867e9f05dbd1ed2fbeeb7368a8f80ddd6c31c0bd8349050810ef8c16"} Jul 11 08:07:40.677231 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:40.677154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" event={"ID":"3b9e033a-7b0a-408b-8100-374fed544d8f","Type":"ContainerStarted","Data":"9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525"} Jul 11 08:07:40.677311 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:40.677289 2572 scope.go:117] "RemoveContainer" containerID="b8e756f8867e9f05dbd1ed2fbeeb7368a8f80ddd6c31c0bd8349050810ef8c16" Jul 11 08:07:41.684297 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:41.684262 2572 generic.go:356] "Generic (PLEG): container finished" podID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerID="d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d" exitCode=7 Jul 11 08:07:41.684748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:41.684312 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" event={"ID":"3b9e033a-7b0a-408b-8100-374fed544d8f","Type":"ContainerDied","Data":"d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d"} Jul 11 08:07:41.684748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:41.684359 2572 scope.go:117] "RemoveContainer" containerID="b8e756f8867e9f05dbd1ed2fbeeb7368a8f80ddd6c31c0bd8349050810ef8c16" Jul 11 08:07:41.684748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:41.684540 2572 scope.go:117] "RemoveContainer" containerID="d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d" Jul 11 08:07:41.684874 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:07:41.684804 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5_odh-ai-gateway-infra(3b9e033a-7b0a-408b-8100-374fed544d8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" Jul 11 08:07:42.080900 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:42.080865 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:07:42.699101 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:42.699073 2572 scope.go:117] "RemoveContainer" containerID="d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d" Jul 11 08:07:42.699491 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:07:42.699245 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5_odh-ai-gateway-infra(3b9e033a-7b0a-408b-8100-374fed544d8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" Jul 11 08:07:50.749470 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.749436 2572 generic.go:356] "Generic (PLEG): container finished" podID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerID="4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365" exitCode=0 Jul 11 08:07:50.749919 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.749505 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerDied","Data":"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365"} Jul 11 08:07:50.749963 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.749921 2572 scope.go:117] "RemoveContainer" containerID="4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365" Jul 11 08:07:50.750988 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.750959 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:50.751075 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.751057 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:50.751135 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:50.751119 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:07:51.654958 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:51.654914 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:51.761730 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:51.761686 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerStarted","Data":"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be"} Jul 11 08:07:51.985957 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:51.985921 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:51.987039 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:51.987014 2572 patch_prober.go:28] interesting pod/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 11 08:07:51.987171 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:51.987062 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 11 08:07:52.986574 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:52.986538 2572 patch_prober.go:28] interesting pod/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 11 08:07:52.986973 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:52.986603 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 11 08:07:53.985773 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:53.985738 2572 patch_prober.go:28] interesting pod/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 11 08:07:53.985939 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:53.985796 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 11 08:07:54.986016 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:54.985977 2572 patch_prober.go:28] interesting pod/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 11 08:07:54.986427 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:54.986035 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 11 08:07:55.990367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:55.990331 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:55.990854 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:55.990646 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:55.991366 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:55.991348 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:07:56.705079 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:56.705054 2572 scope.go:117] "RemoveContainer" containerID="d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d" Jul 11 08:07:57.795345 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:57.795314 2572 generic.go:356] "Generic (PLEG): container finished" podID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerID="dd04eb6e0148847d15136e9faed74d35bfa938846de400ec2e33c7b025d33dca" exitCode=0 Jul 11 08:07:57.795848 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:57.795391 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" event={"ID":"3b9e033a-7b0a-408b-8100-374fed544d8f","Type":"ContainerDied","Data":"dd04eb6e0148847d15136e9faed74d35bfa938846de400ec2e33c7b025d33dca"} Jul 11 08:07:57.795848 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:57.795457 2572 scope.go:117] "RemoveContainer" containerID="d682fc927c6afcf2753e4ec41c7202f85900f3b42e35ab14e1e3cfef5f2b864d" Jul 11 08:07:58.825930 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:58.825900 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:07:58.937952 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:58.937928 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:59.026388 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.026353 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca\") pod \"3b9e033a-7b0a-408b-8100-374fed544d8f\" (UID: \"3b9e033a-7b0a-408b-8100-374fed544d8f\") " Jul 11 08:07:59.026770 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.026745 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b9e033a-7b0a-408b-8100-374fed544d8f" (UID: "3b9e033a-7b0a-408b-8100-374fed544d8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:07:59.127501 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.127423 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9e033a-7b0a-408b-8100-374fed544d8f-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:07:59.809221 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.809189 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" event={"ID":"3b9e033a-7b0a-408b-8100-374fed544d8f","Type":"ContainerDied","Data":"9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525"} Jul 11 08:07:59.809221 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.809222 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5" Jul 11 08:07:59.809444 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.809235 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e585f17737d64b31a28db726167a8a4d08f3ce30b6b77ae1315749f17f81525" Jul 11 08:07:59.838069 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.838043 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:07:59.840309 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:07:59.840289 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-7kkv5"] Jul 11 08:08:00.712101 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:00.712069 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" path="/var/lib/kubelet/pods/3b9e033a-7b0a-408b-8100-374fed544d8f/volumes" Jul 11 08:08:30.734768 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:30.734730 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:08:30.739899 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:30.739870 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:08:39.417578 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.417538 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4"] Jul 11 08:08:39.417987 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.417940 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.417987 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.417951 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.417987 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.417966 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.417987 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.417971 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.418126 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.418037 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.418126 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.418046 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.418126 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.418051 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:39.420366 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.420350 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:39.422744 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.422717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:08:39.422878 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.422834 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:08:39.427747 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.427717 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4"] Jul 11 08:08:39.499838 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.499805 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:08:39.500085 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.500058 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" containerID="cri-o://0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be" gracePeriod=30 Jul 11 08:08:39.506383 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.506357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4\" (UID: \"85e1ae30-cd33-4f29-af40-d58239a20975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:39.607351 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.607312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4\" (UID: \"85e1ae30-cd33-4f29-af40-d58239a20975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:39.608005 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.607984 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4\" (UID: \"85e1ae30-cd33-4f29-af40-d58239a20975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:39.731842 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.731750 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:39.871493 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:39.871462 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4"] Jul 11 08:08:39.873797 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:08:39.873756 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e1ae30_cd33_4f29_af40_d58239a20975.slice/crio-ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5 WatchSource:0}: Error finding container ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5: Status 404 returned error can't find the container with id ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5 Jul 11 08:08:40.041189 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:40.041154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" event={"ID":"85e1ae30-cd33-4f29-af40-d58239a20975","Type":"ContainerStarted","Data":"ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5"} Jul 11 08:08:40.993578 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:40.993537 2572 patch_prober.go:28] interesting pod/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 11 08:08:40.993967 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:40.993606 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 08:08:41.048326 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:41.048289 2572 generic.go:356] "Generic (PLEG): container finished" podID="85e1ae30-cd33-4f29-af40-d58239a20975" containerID="c5023ac8e6cb7600d57f95d532a25824e33ab02e44a3db717d2ad168e637b826" exitCode=0 Jul 11 08:08:41.048473 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:41.048367 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" event={"ID":"85e1ae30-cd33-4f29-af40-d58239a20975","Type":"ContainerDied","Data":"c5023ac8e6cb7600d57f95d532a25824e33ab02e44a3db717d2ad168e637b826"} Jul 11 08:08:42.194142 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:42.194119 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:42.281232 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:42.281193 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:08:42.329547 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:42.329455 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca\") pod \"85e1ae30-cd33-4f29-af40-d58239a20975\" (UID: \"85e1ae30-cd33-4f29-af40-d58239a20975\") " Jul 11 08:08:42.329870 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:42.329848 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85e1ae30-cd33-4f29-af40-d58239a20975" (UID: "85e1ae30-cd33-4f29-af40-d58239a20975"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:08:42.430713 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:42.430666 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e1ae30-cd33-4f29-af40-d58239a20975-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:43.061032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:43.061000 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" Jul 11 08:08:43.061205 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:43.061029 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4" event={"ID":"85e1ae30-cd33-4f29-af40-d58239a20975","Type":"ContainerDied","Data":"ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5"} Jul 11 08:08:43.061205 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:43.061070 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccca24b7b9a7b3c87285b4ec6f0fa2863498cbe30fec48b4a8d604d044b13ac5" Jul 11 08:08:44.748342 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.748318 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:08:44.853772 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853737 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853782 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853813 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5tg7\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853837 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853872 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853898 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.853965 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853959 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.854289 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.853982 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.854289 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854046 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo\") pod \"f986d49e-ea75-454e-8e21-88e4a984cf5d\" (UID: \"f986d49e-ea75-454e-8e21-88e4a984cf5d\") " Jul 11 08:08:44.854463 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854423 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:08:44.854553 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854483 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:08:44.854810 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854668 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:08:44.854810 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854739 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:08:44.855032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.854827 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:08:44.856825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.856721 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:08:44.856950 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.856862 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7" (OuterVolumeSpecName: "kube-api-access-v5tg7") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "kube-api-access-v5tg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:08:44.857014 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.856979 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:08:44.857076 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.857018 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f986d49e-ea75-454e-8e21-88e4a984cf5d" (UID: "f986d49e-ea75-454e-8e21-88e4a984cf5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 11 08:08:44.955327 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955260 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-envoy\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955327 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955313 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-data\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955327 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955327 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5tg7\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-kube-api-access-v5tg7\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955343 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f986d49e-ea75-454e-8e21-88e4a984cf5d-istiod-ca-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955354 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955363 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955370 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-credential-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955378 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f986d49e-ea75-454e-8e21-88e4a984cf5d-workload-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:44.955566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:44.955387 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f986d49e-ea75-454e-8e21-88e4a984cf5d-istio-podinfo\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:08:45.073642 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.073546 2572 generic.go:356] "Generic (PLEG): container finished" podID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerID="0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be" exitCode=0 Jul 11 08:08:45.073642 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.073613 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" Jul 11 08:08:45.073922 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.073633 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerDied","Data":"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be"} Jul 11 08:08:45.073922 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.073758 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx" event={"ID":"f986d49e-ea75-454e-8e21-88e4a984cf5d","Type":"ContainerDied","Data":"4e620783417891bae2117a95d0ce6c2882fc4e56dada74c8dd50b11ab734d929"} Jul 11 08:08:45.073922 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.073798 2572 scope.go:117] "RemoveContainer" containerID="0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be" Jul 11 08:08:45.084078 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.084062 2572 scope.go:117] "RemoveContainer" containerID="4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365" Jul 11 08:08:45.092330 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.092313 2572 scope.go:117] "RemoveContainer" containerID="0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be" Jul 11 08:08:45.092615 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:08:45.092592 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be\": container with ID starting with 0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be not found: ID does not exist" containerID="0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be" Jul 11 08:08:45.092667 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.092625 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be"} err="failed to get container status \"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be\": rpc error: code = NotFound desc = could not find container \"0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be\": container with ID starting with 0c3fc977ff342418cfb6197e5475dda0f72b7522d01c6c381e1dcfd0b2b808be not found: ID does not exist" Jul 11 08:08:45.092667 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.092642 2572 scope.go:117] "RemoveContainer" containerID="4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365" Jul 11 08:08:45.092883 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:08:45.092864 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365\": container with ID starting with 4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365 not found: ID does not exist" containerID="4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365" Jul 11 08:08:45.092943 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.092891 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365"} err="failed to get container status \"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365\": rpc error: code = NotFound desc = could not find container \"4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365\": container with ID starting with 4e52447daa4cacc14e9cd2f6ce6eee918f6925538ace7cc65022814db4a56365 not found: ID does not exist" Jul 11 08:08:45.098296 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.098273 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:08:45.102542 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:45.102517 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7f7e8a77-gw-openshift-default-54fbfc5959-nbnzx"] Jul 11 08:08:46.711380 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:46.711347 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" path="/var/lib/kubelet/pods/f986d49e-ea75-454e-8e21-88e4a984cf5d/volumes" Jul 11 08:08:52.532766 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.532731 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533208 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533222 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533234 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85e1ae30-cd33-4f29-af40-d58239a20975" containerName="revoke-keys" Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533240 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e1ae30-cd33-4f29-af40-d58239a20975" containerName="revoke-keys" Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533255 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:52.533305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533264 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9e033a-7b0a-408b-8100-374fed544d8f" containerName="revoke-keys" Jul 11 08:08:52.533626 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533329 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:52.533626 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533337 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="85e1ae30-cd33-4f29-af40-d58239a20975" containerName="revoke-keys" Jul 11 08:08:52.533626 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.533345 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:52.538498 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.538477 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.541171 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.541147 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b0adf7bb-serving-cert\"" Jul 11 08:08:52.541300 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.541147 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 11 08:08:52.542728 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.542672 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:08:52.542871 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.542761 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jhxtl\"" Jul 11 08:08:52.546761 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.546722 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:08:52.621051 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.621015 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6959m\" (UniqueName: \"kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.621051 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.621053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.621377 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.621122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.722241 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.722207 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6959m\" (UniqueName: \"kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.722241 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.722240 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.722448 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.722282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.723056 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.723032 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.724989 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.724968 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.732185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.732160 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6959m\" (UniqueName: \"kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m\") pod \"maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.852410 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.852344 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:52.995024 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:52.994998 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:08:52.996889 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:08:52.996861 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91f045f_74eb_4374_b60f_9402e74cfcf5.slice/crio-6e582d3e94a180e2be472fe585bac1d0a4ed4ca2167eaa503cae773e69b990ab WatchSource:0}: Error finding container 6e582d3e94a180e2be472fe585bac1d0a4ed4ca2167eaa503cae773e69b990ab: Status 404 returned error can't find the container with id 6e582d3e94a180e2be472fe585bac1d0a4ed4ca2167eaa503cae773e69b990ab Jul 11 08:08:53.118270 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:53.118181 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" event={"ID":"d91f045f-74eb-4374-b60f-9402e74cfcf5","Type":"ContainerStarted","Data":"6e582d3e94a180e2be472fe585bac1d0a4ed4ca2167eaa503cae773e69b990ab"} Jul 11 08:08:54.125246 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:54.125210 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" event={"ID":"d91f045f-74eb-4374-b60f-9402e74cfcf5","Type":"ContainerStarted","Data":"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2"} Jul 11 08:08:54.125630 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:54.125272 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:08:54.155475 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:54.155426 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" podStartSLOduration=1.665721998 podStartE2EDuration="2.155410558s" totalImagesPullingTime="489.68856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:08:54.153301826 +0000 UTC m=+1823.999290721" watchObservedRunningTime="2026-07-11 08:08:54.155410558 +0000 UTC m=+1824.001399470" Jul 11 08:08:55.185855 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:55.185744 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:08:57.571429 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.571398 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:08:57.571843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.571796 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:57.571843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.571808 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f986d49e-ea75-454e-8e21-88e4a984cf5d" containerName="istio-proxy" Jul 11 08:08:57.575192 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.575172 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:08:57.577951 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.577929 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:08:57.582875 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.582849 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:08:57.662896 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.662848 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7\" (UID: \"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:08:57.763592 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.763544 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7\" (UID: \"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:08:57.764216 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.764199 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7\" (UID: \"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:08:57.887980 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:57.887891 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:08:58.021322 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:58.021296 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:08:58.023510 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:08:58.023484 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ed0ce7_4eb1_41ee_882b_80ab2b6ff0ff.slice/crio-82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8 WatchSource:0}: Error finding container 82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8: Status 404 returned error can't find the container with id 82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8 Jul 11 08:08:58.149139 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:58.149060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerStarted","Data":"82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8"} Jul 11 08:08:59.155969 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:59.155883 2572 generic.go:356] "Generic (PLEG): container finished" podID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerID="ebbdeaf6aae9ab982bb8a4508c3493cbc2e9bf53664096a2de8931a49b2ed4c1" exitCode=7 Jul 11 08:08:59.156304 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:59.155962 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerDied","Data":"ebbdeaf6aae9ab982bb8a4508c3493cbc2e9bf53664096a2de8931a49b2ed4c1"} Jul 11 08:08:59.156304 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:08:59.156172 2572 scope.go:117] "RemoveContainer" containerID="ebbdeaf6aae9ab982bb8a4508c3493cbc2e9bf53664096a2de8931a49b2ed4c1" Jul 11 08:09:00.136277 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:00.136196 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:09:00.164663 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:00.164625 2572 generic.go:356] "Generic (PLEG): container finished" podID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerID="441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563" exitCode=7 Jul 11 08:09:00.165129 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:00.164715 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerDied","Data":"441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563"} Jul 11 08:09:00.165129 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:00.164772 2572 scope.go:117] "RemoveContainer" containerID="ebbdeaf6aae9ab982bb8a4508c3493cbc2e9bf53664096a2de8931a49b2ed4c1" Jul 11 08:09:00.165129 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:00.164994 2572 scope.go:117] "RemoveContainer" containerID="441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563" Jul 11 08:09:00.165287 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:09:00.165240 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7_odh-ai-gateway-infra(f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" Jul 11 08:09:01.172337 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:01.172301 2572 scope.go:117] "RemoveContainer" containerID="441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563" Jul 11 08:09:01.172775 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:09:01.172473 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7_odh-ai-gateway-infra(f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" Jul 11 08:09:15.705331 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:15.705302 2572 scope.go:117] "RemoveContainer" containerID="441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563" Jul 11 08:09:16.256102 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:16.256065 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerStarted","Data":"4087ad2dbdd5b9f5fb41d8f4239b4031a85a68a715e18f9d514d2ee355ae6096"} Jul 11 08:09:16.273742 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:16.273675 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" podStartSLOduration=19.27366025 podStartE2EDuration="19.27366025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:09:16.270317005 +0000 UTC m=+1846.116305899" watchObservedRunningTime="2026-07-11 08:09:16.27366025 +0000 UTC m=+1846.119649121" Jul 11 08:09:17.263098 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:17.263065 2572 generic.go:356] "Generic (PLEG): container finished" podID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerID="4087ad2dbdd5b9f5fb41d8f4239b4031a85a68a715e18f9d514d2ee355ae6096" exitCode=0 Jul 11 08:09:17.263485 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:17.263135 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerDied","Data":"4087ad2dbdd5b9f5fb41d8f4239b4031a85a68a715e18f9d514d2ee355ae6096"} Jul 11 08:09:17.263485 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:17.263197 2572 scope.go:117] "RemoveContainer" containerID="441fd3b0a4fc4a97bb6ecb17d07d78bd46c16e997264dc825c52de20a9e88563" Jul 11 08:09:17.281469 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:17.281445 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:09:18.400307 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:18.400275 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:09:18.543418 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:18.543334 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca\") pod \"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff\" (UID: \"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff\") " Jul 11 08:09:18.543681 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:18.543659 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" (UID: "f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:09:18.644955 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:18.644925 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:09:19.277167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:19.277139 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" Jul 11 08:09:19.277167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:19.277144 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7" event={"ID":"f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff","Type":"ContainerDied","Data":"82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8"} Jul 11 08:09:19.277422 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:19.277190 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82850e98bb71ea0e17eeffc219c807a65736fce2a34d2b55131aa47c4b386ff8" Jul 11 08:09:19.309899 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:19.309870 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:09:19.311976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:19.311956 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-7zml7"] Jul 11 08:09:20.711535 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:20.711503 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" path="/var/lib/kubelet/pods/f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff/volumes" Jul 11 08:09:57.613592 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.613512 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9"] Jul 11 08:09:57.614001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.613911 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.614001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.613924 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.614001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.613934 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.614001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.613940 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.614126 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.614006 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.614126 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.614017 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:09:57.617005 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.616989 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:09:57.619385 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.619362 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:09:57.624379 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.624353 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9"] Jul 11 08:09:57.662237 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.662213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9\" (UID: \"5f21581e-d570-48b0-9730-135657db3b7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:09:57.762832 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.762801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9\" (UID: \"5f21581e-d570-48b0-9730-135657db3b7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:09:57.763435 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.763416 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9\" (UID: \"5f21581e-d570-48b0-9730-135657db3b7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:09:57.928744 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:57.928655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:09:58.055624 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:58.055596 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9"] Jul 11 08:09:58.057652 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:09:58.057621 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f21581e_d570_48b0_9730_135657db3b7b.slice/crio-920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b WatchSource:0}: Error finding container 920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b: Status 404 returned error can't find the container with id 920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b Jul 11 08:09:58.503091 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:58.503002 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" event={"ID":"5f21581e-d570-48b0-9730-135657db3b7b","Type":"ContainerStarted","Data":"ca0772c3a1e43fd070b2c4389cc223dc5db6bf97cf0b0a3f7541b439c9159294"} Jul 11 08:09:58.503091 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:58.503054 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" event={"ID":"5f21581e-d570-48b0-9730-135657db3b7b","Type":"ContainerStarted","Data":"920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b"} Jul 11 08:09:58.518663 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:58.518617 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" podStartSLOduration=1.5186029109999999 podStartE2EDuration="1.518602911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:09:58.51641601 +0000 UTC m=+1888.362404904" watchObservedRunningTime="2026-07-11 08:09:58.518602911 +0000 UTC m=+1888.364591831" Jul 11 08:09:59.509777 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:59.509742 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f21581e-d570-48b0-9730-135657db3b7b" containerID="ca0772c3a1e43fd070b2c4389cc223dc5db6bf97cf0b0a3f7541b439c9159294" exitCode=0 Jul 11 08:09:59.510155 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:09:59.509871 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" event={"ID":"5f21581e-d570-48b0-9730-135657db3b7b","Type":"ContainerDied","Data":"ca0772c3a1e43fd070b2c4389cc223dc5db6bf97cf0b0a3f7541b439c9159294"} Jul 11 08:10:00.648178 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:00.648153 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:10:00.687391 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:00.687351 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca\") pod \"5f21581e-d570-48b0-9730-135657db3b7b\" (UID: \"5f21581e-d570-48b0-9730-135657db3b7b\") " Jul 11 08:10:00.687697 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:00.687674 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f21581e-d570-48b0-9730-135657db3b7b" (UID: "5f21581e-d570-48b0-9730-135657db3b7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:10:00.788358 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:00.788268 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21581e-d570-48b0-9730-135657db3b7b-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:10:01.522914 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:01.522880 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" Jul 11 08:10:01.522914 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:01.522902 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9" event={"ID":"5f21581e-d570-48b0-9730-135657db3b7b","Type":"ContainerDied","Data":"920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b"} Jul 11 08:10:01.523138 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:01.522977 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="920fbb151638721e6a5ba3ca8fe9994069d50a8650be2573edf3cdd368a4c89b" Jul 11 08:10:07.961370 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:07.961335 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:10:07.961999 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:07.961599 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" podUID="d91f045f-74eb-4374-b60f-9402e74cfcf5" containerName="maas-api" containerID="cri-o://a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2" gracePeriod=30 Jul 11 08:10:08.220385 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.220329 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:10:08.256591 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.256559 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls\") pod \"d91f045f-74eb-4374-b60f-9402e74cfcf5\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " Jul 11 08:10:08.256748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.256623 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca\") pod \"d91f045f-74eb-4374-b60f-9402e74cfcf5\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " Jul 11 08:10:08.256748 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.256731 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6959m\" (UniqueName: \"kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m\") pod \"d91f045f-74eb-4374-b60f-9402e74cfcf5\" (UID: \"d91f045f-74eb-4374-b60f-9402e74cfcf5\") " Jul 11 08:10:08.256959 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.256937 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d91f045f-74eb-4374-b60f-9402e74cfcf5" (UID: "d91f045f-74eb-4374-b60f-9402e74cfcf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:10:08.258729 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.258681 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d91f045f-74eb-4374-b60f-9402e74cfcf5" (UID: "d91f045f-74eb-4374-b60f-9402e74cfcf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 08:10:08.258836 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.258790 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m" (OuterVolumeSpecName: "kube-api-access-6959m") pod "d91f045f-74eb-4374-b60f-9402e74cfcf5" (UID: "d91f045f-74eb-4374-b60f-9402e74cfcf5"). InnerVolumeSpecName "kube-api-access-6959m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:10:08.358091 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.358062 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6959m\" (UniqueName: \"kubernetes.io/projected/d91f045f-74eb-4374-b60f-9402e74cfcf5-kube-api-access-6959m\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:10:08.358091 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.358086 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91f045f-74eb-4374-b60f-9402e74cfcf5-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:10:08.358091 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.358095 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91f045f-74eb-4374-b60f-9402e74cfcf5-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:10:08.567198 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.567159 2572 generic.go:356] "Generic (PLEG): container finished" podID="d91f045f-74eb-4374-b60f-9402e74cfcf5" containerID="a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2" exitCode=0 Jul 11 08:10:08.567368 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.567206 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" Jul 11 08:10:08.567368 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.567236 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" event={"ID":"d91f045f-74eb-4374-b60f-9402e74cfcf5","Type":"ContainerDied","Data":"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2"} Jul 11 08:10:08.567368 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.567307 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn" event={"ID":"d91f045f-74eb-4374-b60f-9402e74cfcf5","Type":"ContainerDied","Data":"6e582d3e94a180e2be472fe585bac1d0a4ed4ca2167eaa503cae773e69b990ab"} Jul 11 08:10:08.567368 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.567329 2572 scope.go:117] "RemoveContainer" containerID="a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2" Jul 11 08:10:08.577214 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.577195 2572 scope.go:117] "RemoveContainer" containerID="a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2" Jul 11 08:10:08.577478 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:10:08.577460 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2\": container with ID starting with a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2 not found: ID does not exist" containerID="a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2" Jul 11 08:10:08.577533 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.577485 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2"} err="failed to get container status \"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2\": rpc error: code = NotFound desc = could not find container \"a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2\": container with ID starting with a3f58f9a3861c2d6c0c0da792a9ae4ea93d2f955f2d117c49a179d63979c70a2 not found: ID does not exist" Jul 11 08:10:08.588426 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.588377 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:10:08.591275 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.591253 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0adf7bb-767579ccb9-5q5fn"] Jul 11 08:10:08.711562 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:08.711529 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91f045f-74eb-4374-b60f-9402e74cfcf5" path="/var/lib/kubelet/pods/d91f045f-74eb-4374-b60f-9402e74cfcf5/volumes" Jul 11 08:10:10.887008 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:10.886975 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:10:23.942778 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.942738 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:10:23.943264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943246 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91f045f-74eb-4374-b60f-9402e74cfcf5" containerName="maas-api" Jul 11 08:10:23.943264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943266 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91f045f-74eb-4374-b60f-9402e74cfcf5" containerName="maas-api" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943273 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943278 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943293 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21581e-d570-48b0-9730-135657db3b7b" containerName="revoke-keys" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943299 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21581e-d570-48b0-9730-135657db3b7b" containerName="revoke-keys" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943377 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ed0ce7-4eb1-41ee-882b-80ab2b6ff0ff" containerName="revoke-keys" Jul 11 08:10:23.943396 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943395 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91f045f-74eb-4374-b60f-9402e74cfcf5" containerName="maas-api" Jul 11 08:10:23.943611 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.943404 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21581e-d570-48b0-9730-135657db3b7b" containerName="revoke-keys" Jul 11 08:10:23.947732 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.947686 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:23.950676 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.950657 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c12c06b3-openshift-default-dockercfg-mjcr2\"" Jul 11 08:10:23.957925 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:23.957834 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:10:24.097394 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097394 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097397 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097776 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097618 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097776 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.097776 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.097666 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjd7t\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198726 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198626 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198726 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjd7t\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.198924 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.198908 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199236 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199011 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199236 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199111 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199236 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199141 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199384 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199363 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199446 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199431 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.199750 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.199726 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.201102 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.201078 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.201274 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.201254 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.207232 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.207204 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjd7t\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.207232 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.207214 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token\") pod \"e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.261861 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.261840 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:24.394562 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.394537 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:10:24.395918 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:10:24.395888 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90bcedb_c0ee_4c7b_af81_f478f60ba144.slice/crio-f48f346c12ebcbf1496629004d9cdfc232c6493be5d3498e2aa2df71e2fce1f9 WatchSource:0}: Error finding container f48f346c12ebcbf1496629004d9cdfc232c6493be5d3498e2aa2df71e2fce1f9: Status 404 returned error can't find the container with id f48f346c12ebcbf1496629004d9cdfc232c6493be5d3498e2aa2df71e2fce1f9 Jul 11 08:10:24.398066 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.398036 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:24.398127 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.398099 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:24.398172 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.398135 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:24.659259 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.659223 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerStarted","Data":"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75"} Jul 11 08:10:24.659464 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.659310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerStarted","Data":"f48f346c12ebcbf1496629004d9cdfc232c6493be5d3498e2aa2df71e2fce1f9"} Jul 11 08:10:24.676463 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:24.676389 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podStartSLOduration=1.6763751770000002 podStartE2EDuration="1.676375177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:10:24.675537289 +0000 UTC m=+1914.521526255" watchObservedRunningTime="2026-07-11 08:10:24.676375177 +0000 UTC m=+1914.522364048" Jul 11 08:10:25.262278 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:25.262243 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:25.267130 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:25.267110 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:25.666087 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:25.665989 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:25.667090 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:25.667070 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:29.456016 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.455983 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:29.459600 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.459582 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:29.462295 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.462256 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:10:29.462447 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.462404 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:10:29.466943 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.466917 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:29.559687 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.559652 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-96v62\" (UID: \"ab27cba8-d92f-4088-b463-b20c73986363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:29.660233 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.660202 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-96v62\" (UID: \"ab27cba8-d92f-4088-b463-b20c73986363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:29.660852 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.660832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-96v62\" (UID: \"ab27cba8-d92f-4088-b463-b20c73986363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:29.772612 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.772584 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:29.902585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:29.902551 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:29.903957 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:10:29.903930 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab27cba8_d92f_4088_b463_b20c73986363.slice/crio-3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11 WatchSource:0}: Error finding container 3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11: Status 404 returned error can't find the container with id 3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11 Jul 11 08:10:30.694320 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:30.694287 2572 generic.go:356] "Generic (PLEG): container finished" podID="ab27cba8-d92f-4088-b463-b20c73986363" containerID="8a7961bcf10f2f0c8ae7251070f7136d77bfd5177985132b75494c616f5198eb" exitCode=7 Jul 11 08:10:30.694679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:30.694328 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" event={"ID":"ab27cba8-d92f-4088-b463-b20c73986363","Type":"ContainerDied","Data":"8a7961bcf10f2f0c8ae7251070f7136d77bfd5177985132b75494c616f5198eb"} Jul 11 08:10:30.694679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:30.694374 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" event={"ID":"ab27cba8-d92f-4088-b463-b20c73986363","Type":"ContainerStarted","Data":"3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11"} Jul 11 08:10:30.694679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:30.694577 2572 scope.go:117] "RemoveContainer" containerID="8a7961bcf10f2f0c8ae7251070f7136d77bfd5177985132b75494c616f5198eb" Jul 11 08:10:31.702077 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:31.701991 2572 generic.go:356] "Generic (PLEG): container finished" podID="ab27cba8-d92f-4088-b463-b20c73986363" containerID="9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438" exitCode=7 Jul 11 08:10:31.702077 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:31.702041 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" event={"ID":"ab27cba8-d92f-4088-b463-b20c73986363","Type":"ContainerDied","Data":"9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438"} Jul 11 08:10:31.702574 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:31.702087 2572 scope.go:117] "RemoveContainer" containerID="8a7961bcf10f2f0c8ae7251070f7136d77bfd5177985132b75494c616f5198eb" Jul 11 08:10:31.702574 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:31.702304 2572 scope.go:117] "RemoveContainer" containerID="9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438" Jul 11 08:10:31.702574 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:10:31.702529 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c12c06b3-96v62_odh-ai-gateway-infra(ab27cba8-d92f-4088-b463-b20c73986363)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" podUID="ab27cba8-d92f-4088-b463-b20c73986363" Jul 11 08:10:32.486000 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:32.485965 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:10:32.709066 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:32.709040 2572 scope.go:117] "RemoveContainer" containerID="9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438" Jul 11 08:10:32.709430 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:10:32.709198 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c12c06b3-96v62_odh-ai-gateway-infra(ab27cba8-d92f-4088-b463-b20c73986363)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" podUID="ab27cba8-d92f-4088-b463-b20c73986363" Jul 11 08:10:40.666880 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.666843 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 11 08:10:40.667292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.666901 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 11 08:10:40.758175 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.758138 2572 generic.go:356] "Generic (PLEG): container finished" podID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerID="ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75" exitCode=0 Jul 11 08:10:40.758335 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.758212 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerDied","Data":"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75"} Jul 11 08:10:40.758606 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.758586 2572 scope.go:117] "RemoveContainer" containerID="ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75" Jul 11 08:10:40.759642 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.759610 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:40.759740 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.759697 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:40.759794 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:40.759775 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:10:41.766084 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:41.766048 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerStarted","Data":"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba"} Jul 11 08:10:42.262264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:42.262226 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:42.262468 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:42.262438 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 11 08:10:42.262537 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:42.262495 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 11 08:10:43.262783 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:43.262748 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 11 08:10:43.263148 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:43.262805 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 11 08:10:43.705968 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:43.705885 2572 scope.go:117] "RemoveContainer" containerID="9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438" Jul 11 08:10:44.262329 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:44.262293 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 11 08:10:44.262482 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:44.262347 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 11 08:10:44.784119 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:44.784092 2572 generic.go:356] "Generic (PLEG): container finished" podID="ab27cba8-d92f-4088-b463-b20c73986363" containerID="e14b491b88e76caaea3ecf00e029ccb9a409ce2be24ec8fcd17db4228c2501e0" exitCode=0 Jul 11 08:10:44.784425 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:44.784157 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" event={"ID":"ab27cba8-d92f-4088-b463-b20c73986363","Type":"ContainerDied","Data":"e14b491b88e76caaea3ecf00e029ccb9a409ce2be24ec8fcd17db4228c2501e0"} Jul 11 08:10:44.784425 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:44.784213 2572 scope.go:117] "RemoveContainer" containerID="9e6ff430410ce90d09479f44822647f6bd75250f9b581277cb3cf743e1363438" Jul 11 08:10:45.262715 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.262677 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 11 08:10:45.262901 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.262773 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 11 08:10:45.816007 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.815965 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:45.919502 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.919473 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:45.993357 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.993326 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca\") pod \"ab27cba8-d92f-4088-b463-b20c73986363\" (UID: \"ab27cba8-d92f-4088-b463-b20c73986363\") " Jul 11 08:10:45.993678 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:45.993655 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab27cba8-d92f-4088-b463-b20c73986363" (UID: "ab27cba8-d92f-4088-b463-b20c73986363"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:10:46.094756 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.094653 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab27cba8-d92f-4088-b463-b20c73986363-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:10:46.266588 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.266557 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:46.266893 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.266873 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:46.267432 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.267414 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:10:46.798427 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.798403 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" Jul 11 08:10:46.798609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.798442 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62" event={"ID":"ab27cba8-d92f-4088-b463-b20c73986363","Type":"ContainerDied","Data":"3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11"} Jul 11 08:10:46.798609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.798493 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ed6e49c6aa786c33523a2ad8728241416fd240e5287fd4b08ccbebb73f3ff11" Jul 11 08:10:46.826104 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.826073 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:46.828317 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:46.828297 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-96v62"] Jul 11 08:10:48.710975 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:10:48.710946 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab27cba8-d92f-4088-b463-b20c73986363" path="/var/lib/kubelet/pods/ab27cba8-d92f-4088-b463-b20c73986363/volumes" Jul 11 08:11:19.495258 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495215 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq"] Jul 11 08:11:19.495879 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495772 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.495879 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495795 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.495879 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495823 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.495879 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495832 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.496100 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495942 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.496100 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.495954 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:11:19.499252 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.499230 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:19.502033 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.501907 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:11:19.502989 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.502964 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:11:19.505129 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.505107 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq"] Jul 11 08:11:19.591824 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.591794 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq\" (UID: \"22af16a3-15fd-43d9-8d75-a23d2636dc15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:19.692769 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.692741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq\" (UID: \"22af16a3-15fd-43d9-8d75-a23d2636dc15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:19.693334 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.693316 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq\" (UID: \"22af16a3-15fd-43d9-8d75-a23d2636dc15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:19.811278 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.811190 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:19.940806 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.940776 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq"] Jul 11 08:11:19.941549 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:11:19.941522 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22af16a3_15fd_43d9_8d75_a23d2636dc15.slice/crio-461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa WatchSource:0}: Error finding container 461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa: Status 404 returned error can't find the container with id 461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa Jul 11 08:11:19.986603 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:19.986576 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" event={"ID":"22af16a3-15fd-43d9-8d75-a23d2636dc15","Type":"ContainerStarted","Data":"461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa"} Jul 11 08:11:20.993018 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:20.992982 2572 generic.go:356] "Generic (PLEG): container finished" podID="22af16a3-15fd-43d9-8d75-a23d2636dc15" containerID="229de635ecccbf996b1b437d2b7d90691fe651c71230a75819174f602c0a71a5" exitCode=0 Jul 11 08:11:20.993397 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:20.993067 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" event={"ID":"22af16a3-15fd-43d9-8d75-a23d2636dc15","Type":"ContainerDied","Data":"229de635ecccbf996b1b437d2b7d90691fe651c71230a75819174f602c0a71a5"} Jul 11 08:11:22.134219 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:22.134196 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:22.314863 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:22.314757 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca\") pod \"22af16a3-15fd-43d9-8d75-a23d2636dc15\" (UID: \"22af16a3-15fd-43d9-8d75-a23d2636dc15\") " Jul 11 08:11:22.315101 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:22.315076 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22af16a3-15fd-43d9-8d75-a23d2636dc15" (UID: "22af16a3-15fd-43d9-8d75-a23d2636dc15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:11:22.415731 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:22.415679 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22af16a3-15fd-43d9-8d75-a23d2636dc15-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:23.007059 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:23.007031 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" Jul 11 08:11:23.007234 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:23.007057 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq" event={"ID":"22af16a3-15fd-43d9-8d75-a23d2636dc15","Type":"ContainerDied","Data":"461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa"} Jul 11 08:11:23.007234 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:23.007099 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="461fdfc69b4c756196453b628c80995a999b4428fb191e6c6320911cecc225aa" Jul 11 08:11:32.489758 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:32.489720 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:11:44.740334 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:44.740296 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:11:44.740797 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:44.740500 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" containerID="cri-o://a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba" gracePeriod=30 Jul 11 08:11:46.269557 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:46.269518 2572 patch_prober.go:28] interesting pod/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 11 08:11:46.270004 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:46.269593 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 08:11:49.987185 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:49.987160 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:11:50.050417 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050388 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050423 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050470 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050501 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050580 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050540 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050601 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050655 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050678 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.050819 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050732 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjd7t\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t\") pod \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\" (UID: \"d90bcedb-c0ee-4c7b-af81-f478f60ba144\") " Jul 11 08:11:50.051022 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.050917 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:11:50.051104 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.051076 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:11:50.051168 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.051117 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istiod-ca-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.051585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.051502 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:11:50.051585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.051515 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data" (OuterVolumeSpecName: "istio-data") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:11:50.051585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.051572 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:11:50.052855 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.052827 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token" (OuterVolumeSpecName: "istio-token") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:11:50.053030 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.053007 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 11 08:11:50.053303 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.053284 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:11:50.053354 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.053338 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t" (OuterVolumeSpecName: "kube-api-access-mjd7t") pod "d90bcedb-c0ee-4c7b-af81-f478f60ba144" (UID: "d90bcedb-c0ee-4c7b-af81-f478f60ba144"). InnerVolumeSpecName "kube-api-access-mjd7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:11:50.152146 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152112 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152146 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152140 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-data\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152146 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152149 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152371 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152159 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-credential-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152371 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152169 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-podinfo\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152371 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152176 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-istio-envoy\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152371 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152183 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d90bcedb-c0ee-4c7b-af81-f478f60ba144-workload-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.152371 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.152191 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjd7t\" (UniqueName: \"kubernetes.io/projected/d90bcedb-c0ee-4c7b-af81-f478f60ba144-kube-api-access-mjd7t\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:11:50.169558 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.169525 2572 generic.go:356] "Generic (PLEG): container finished" podID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerID="a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba" exitCode=0 Jul 11 08:11:50.169734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.169580 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" Jul 11 08:11:50.169734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.169610 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerDied","Data":"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba"} Jul 11 08:11:50.169734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.169696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x" event={"ID":"d90bcedb-c0ee-4c7b-af81-f478f60ba144","Type":"ContainerDied","Data":"f48f346c12ebcbf1496629004d9cdfc232c6493be5d3498e2aa2df71e2fce1f9"} Jul 11 08:11:50.169917 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.169742 2572 scope.go:117] "RemoveContainer" containerID="a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba" Jul 11 08:11:50.179403 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.179387 2572 scope.go:117] "RemoveContainer" containerID="ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75" Jul 11 08:11:50.187577 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.187540 2572 scope.go:117] "RemoveContainer" containerID="a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba" Jul 11 08:11:50.188265 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:11:50.187968 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba\": container with ID starting with a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba not found: ID does not exist" containerID="a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba" Jul 11 08:11:50.188265 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.188005 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba"} err="failed to get container status \"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba\": rpc error: code = NotFound desc = could not find container \"a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba\": container with ID starting with a145cfb684bba0de0146d1fdbdbbaf3b4e67b061600b3eec4a55f6cd1db202ba not found: ID does not exist" Jul 11 08:11:50.188265 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.188022 2572 scope.go:117] "RemoveContainer" containerID="ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75" Jul 11 08:11:50.188459 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:11:50.188286 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75\": container with ID starting with ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75 not found: ID does not exist" containerID="ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75" Jul 11 08:11:50.188459 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.188313 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75"} err="failed to get container status \"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75\": rpc error: code = NotFound desc = could not find container \"ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75\": container with ID starting with ab4ca3216cc1cef46b0db843b8965350ae177e32fdd1b6914a8d2fe660025e75 not found: ID does not exist" Jul 11 08:11:50.191134 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.191109 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:11:50.194211 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.194193 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c12c06b3-openshift-default-595cd9c887-hq58x"] Jul 11 08:11:50.712561 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:11:50.712530 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" path="/var/lib/kubelet/pods/d90bcedb-c0ee-4c7b-af81-f478f60ba144/volumes" Jul 11 08:13:08.187995 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:08.187911 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:13:16.994544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:16.994507 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:13:30.773459 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:30.773428 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:13:30.779461 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:30.779442 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:13:30.827292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:30.827270 2572 scope.go:117] "RemoveContainer" containerID="d519e0f2bd9fa1261a199d651bb6f1539871e47b0cfa518cf0ba7922daf64a20" Jul 11 08:13:31.877107 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:31.877073 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:13:35.846211 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846178 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp"] Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846549 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846560 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846570 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846576 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846582 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22af16a3-15fd-43d9-8d75-a23d2636dc15" containerName="revoke-keys" Jul 11 08:13:35.846589 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846587 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="22af16a3-15fd-43d9-8d75-a23d2636dc15" containerName="revoke-keys" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846607 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846617 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846675 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846683 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="22af16a3-15fd-43d9-8d75-a23d2636dc15" containerName="revoke-keys" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846690 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90bcedb-c0ee-4c7b-af81-f478f60ba144" containerName="istio-proxy" Jul 11 08:13:35.846813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.846698 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab27cba8-d92f-4088-b463-b20c73986363" containerName="revoke-keys" Jul 11 08:13:35.855799 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.855765 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:35.858412 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.858390 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:13:35.858558 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.858447 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:13:35.858625 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.858588 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp"] Jul 11 08:13:35.970669 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:35.970621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp\" (UID: \"65f62545-f739-4a3f-bd46-730edb97f453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:36.071445 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.071413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp\" (UID: \"65f62545-f739-4a3f-bd46-730edb97f453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:36.072049 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.072031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp\" (UID: \"65f62545-f739-4a3f-bd46-730edb97f453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:36.167037 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.166956 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:36.298662 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.298630 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp"] Jul 11 08:13:36.299749 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:13:36.299721 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f62545_f739_4a3f_bd46_730edb97f453.slice/crio-a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331 WatchSource:0}: Error finding container a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331: Status 404 returned error can't find the container with id a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331 Jul 11 08:13:36.784494 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.784461 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" event={"ID":"65f62545-f739-4a3f-bd46-730edb97f453","Type":"ContainerStarted","Data":"717060a8cf8fd1cf08c2c220df8c6662b34964d552d94be58df64414138119e6"} Jul 11 08:13:36.784689 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.784509 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" event={"ID":"65f62545-f739-4a3f-bd46-730edb97f453","Type":"ContainerStarted","Data":"a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331"} Jul 11 08:13:36.801197 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:36.801104 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" podStartSLOduration=1.801090144 podStartE2EDuration="1.801090144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:13:36.798237508 +0000 UTC m=+2106.644226402" watchObservedRunningTime="2026-07-11 08:13:36.801090144 +0000 UTC m=+2106.647079036" Jul 11 08:13:37.792623 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:37.792589 2572 generic.go:356] "Generic (PLEG): container finished" podID="65f62545-f739-4a3f-bd46-730edb97f453" containerID="717060a8cf8fd1cf08c2c220df8c6662b34964d552d94be58df64414138119e6" exitCode=0 Jul 11 08:13:37.793018 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:37.792632 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" event={"ID":"65f62545-f739-4a3f-bd46-730edb97f453","Type":"ContainerDied","Data":"717060a8cf8fd1cf08c2c220df8c6662b34964d552d94be58df64414138119e6"} Jul 11 08:13:38.928725 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:38.928685 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:38.996955 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:38.996927 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca\") pod \"65f62545-f739-4a3f-bd46-730edb97f453\" (UID: \"65f62545-f739-4a3f-bd46-730edb97f453\") " Jul 11 08:13:38.997246 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:38.997225 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65f62545-f739-4a3f-bd46-730edb97f453" (UID: "65f62545-f739-4a3f-bd46-730edb97f453"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:13:39.098657 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:39.098580 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f62545-f739-4a3f-bd46-730edb97f453-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:13:39.805059 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:39.805025 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" Jul 11 08:13:39.805059 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:39.805047 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp" event={"ID":"65f62545-f739-4a3f-bd46-730edb97f453","Type":"ContainerDied","Data":"a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331"} Jul 11 08:13:39.805368 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:39.805093 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a37e06f9f708ec930f30c34b071e66c5f9a55873a730429892998de5f03e9331" Jul 11 08:13:49.195088 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:13:49.195047 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:14:30.285245 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:14:30.285164 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:14:30.848566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:14:30.848532 2572 scope.go:117] "RemoveContainer" containerID="dd04eb6e0148847d15136e9faed74d35bfa938846de400ec2e33c7b025d33dca" Jul 11 08:14:41.188256 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:14:41.188223 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:15:00.129310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.129270 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk"] Jul 11 08:15:00.129934 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.129908 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f62545-f739-4a3f-bd46-730edb97f453" containerName="revoke-keys" Jul 11 08:15:00.129934 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.129934 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f62545-f739-4a3f-bd46-730edb97f453" containerName="revoke-keys" Jul 11 08:15:00.130098 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.130067 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f62545-f739-4a3f-bd46-730edb97f453" containerName="revoke-keys" Jul 11 08:15:00.133499 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.133479 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" Jul 11 08:15:00.133616 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.133474 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww"] Jul 11 08:15:00.135939 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.135876 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:15:00.137240 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.137183 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" Jul 11 08:15:00.139454 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.139425 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk"] Jul 11 08:15:00.143110 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.143084 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww"] Jul 11 08:15:00.149778 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.149757 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" Jul 11 08:15:00.155969 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.155952 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" Jul 11 08:15:00.499969 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.499943 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk"] Jul 11 08:15:00.501533 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:15:00.501503 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd9c9de_b961_41eb_ad7d_3abf74f5206b.slice/crio-5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410 WatchSource:0}: Error finding container 5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410: Status 404 returned error can't find the container with id 5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410 Jul 11 08:15:00.523859 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:00.523837 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww"] Jul 11 08:15:00.524896 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:15:00.524864 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8b607b_ca76_47cc_a29c_b57e5965dd63.slice/crio-4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d WatchSource:0}: Error finding container 4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d: Status 404 returned error can't find the container with id 4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d Jul 11 08:15:01.261190 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.261158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" event={"ID":"9a8b607b-ca76-47cc-a29c-b57e5965dd63","Type":"ContainerStarted","Data":"c3259a49ab8cdddb4de85f96593dc3c30207758b2ac24fa1c7658b863b5995cf"} Jul 11 08:15:01.261699 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.261243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" event={"ID":"9a8b607b-ca76-47cc-a29c-b57e5965dd63","Type":"ContainerStarted","Data":"4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d"} Jul 11 08:15:01.262915 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.262889 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" event={"ID":"3cd9c9de-b961-41eb-ad7d-3abf74f5206b","Type":"ContainerStarted","Data":"2fb13d9f50fe9d2fb7b7fe9fb71ce0d37255c271837e8ed27730c912678b7c40"} Jul 11 08:15:01.263029 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.262943 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" event={"ID":"3cd9c9de-b961-41eb-ad7d-3abf74f5206b","Type":"ContainerStarted","Data":"5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410"} Jul 11 08:15:01.279326 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.279279 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" podStartSLOduration=1.279266219 podStartE2EDuration="1.279266219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:15:01.276949342 +0000 UTC m=+2191.122938234" watchObservedRunningTime="2026-07-11 08:15:01.279266219 +0000 UTC m=+2191.125255090" Jul 11 08:15:01.293640 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:01.293591 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" podStartSLOduration=1.293575008 podStartE2EDuration="1.293575008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:15:01.292742204 +0000 UTC m=+2191.138731094" watchObservedRunningTime="2026-07-11 08:15:01.293575008 +0000 UTC m=+2191.139563900" Jul 11 08:15:02.270170 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:02.270136 2572 generic.go:356] "Generic (PLEG): container finished" podID="3cd9c9de-b961-41eb-ad7d-3abf74f5206b" containerID="2fb13d9f50fe9d2fb7b7fe9fb71ce0d37255c271837e8ed27730c912678b7c40" exitCode=0 Jul 11 08:15:02.270590 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:02.270217 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" event={"ID":"3cd9c9de-b961-41eb-ad7d-3abf74f5206b","Type":"ContainerDied","Data":"2fb13d9f50fe9d2fb7b7fe9fb71ce0d37255c271837e8ed27730c912678b7c40"} Jul 11 08:15:02.271690 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:02.271667 2572 generic.go:356] "Generic (PLEG): container finished" podID="9a8b607b-ca76-47cc-a29c-b57e5965dd63" containerID="c3259a49ab8cdddb4de85f96593dc3c30207758b2ac24fa1c7658b863b5995cf" exitCode=0 Jul 11 08:15:02.271830 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:02.271719 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" event={"ID":"9a8b607b-ca76-47cc-a29c-b57e5965dd63","Type":"ContainerDied","Data":"c3259a49ab8cdddb4de85f96593dc3c30207758b2ac24fa1c7658b863b5995cf"} Jul 11 08:15:03.441066 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:03.440795 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" Jul 11 08:15:03.477862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:03.477835 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" Jul 11 08:15:04.285473 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.285443 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" Jul 11 08:15:04.285657 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.285481 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk" event={"ID":"3cd9c9de-b961-41eb-ad7d-3abf74f5206b","Type":"ContainerDied","Data":"5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410"} Jul 11 08:15:04.285657 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.285530 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5332de8baa8b45107af004780e56277818b03d6722d41343386bf49c6f9ef410" Jul 11 08:15:04.286825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.286804 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" event={"ID":"9a8b607b-ca76-47cc-a29c-b57e5965dd63","Type":"ContainerDied","Data":"4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d"} Jul 11 08:15:04.286942 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.286836 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a775cfcbb73cca9b03c409cc633a860f3c07b0106cd3876e00d15e498149b0d" Jul 11 08:15:04.286942 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:04.286839 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29729295-krkww" Jul 11 08:15:05.085350 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:05.085315 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:15:11.290861 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.290829 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m"] Jul 11 08:15:11.291238 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291200 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cd9c9de-b961-41eb-ad7d-3abf74f5206b" containerName="cleanup" Jul 11 08:15:11.291238 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291210 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd9c9de-b961-41eb-ad7d-3abf74f5206b" containerName="cleanup" Jul 11 08:15:11.291238 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291221 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8b607b-ca76-47cc-a29c-b57e5965dd63" containerName="cleanup" Jul 11 08:15:11.291238 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291226 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8b607b-ca76-47cc-a29c-b57e5965dd63" containerName="cleanup" Jul 11 08:15:11.291364 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291285 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a8b607b-ca76-47cc-a29c-b57e5965dd63" containerName="cleanup" Jul 11 08:15:11.291364 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.291293 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cd9c9de-b961-41eb-ad7d-3abf74f5206b" containerName="cleanup" Jul 11 08:15:11.294513 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.294498 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:11.297544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.297518 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:15:11.298437 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.298418 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:15:11.300872 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.300846 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m"] Jul 11 08:15:11.427250 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.427218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m\" (UID: \"666db48b-931c-44a9-80a4-b6f05b96ecbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:11.528615 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.528562 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m\" (UID: \"666db48b-931c-44a9-80a4-b6f05b96ecbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:11.529178 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.529156 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m\" (UID: \"666db48b-931c-44a9-80a4-b6f05b96ecbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:11.606834 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.606764 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:11.749596 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:11.749561 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m"] Jul 11 08:15:11.750799 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:15:11.750768 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666db48b_931c_44a9_80a4_b6f05b96ecbc.slice/crio-6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b WatchSource:0}: Error finding container 6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b: Status 404 returned error can't find the container with id 6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b Jul 11 08:15:12.335074 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:12.335040 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" event={"ID":"666db48b-931c-44a9-80a4-b6f05b96ecbc","Type":"ContainerStarted","Data":"42220f340592baa624a0407dbd37bf397c14fd048844f57d655bef18af8bb0e5"} Jul 11 08:15:12.335452 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:12.335097 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" event={"ID":"666db48b-931c-44a9-80a4-b6f05b96ecbc","Type":"ContainerStarted","Data":"6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b"} Jul 11 08:15:12.351372 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:12.351326 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" podStartSLOduration=1.351311934 podStartE2EDuration="1.351311934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:15:12.349289814 +0000 UTC m=+2202.195278708" watchObservedRunningTime="2026-07-11 08:15:12.351311934 +0000 UTC m=+2202.197300828" Jul 11 08:15:13.342158 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:13.342122 2572 generic.go:356] "Generic (PLEG): container finished" podID="666db48b-931c-44a9-80a4-b6f05b96ecbc" containerID="42220f340592baa624a0407dbd37bf397c14fd048844f57d655bef18af8bb0e5" exitCode=0 Jul 11 08:15:13.342502 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:13.342170 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" event={"ID":"666db48b-931c-44a9-80a4-b6f05b96ecbc","Type":"ContainerDied","Data":"42220f340592baa624a0407dbd37bf397c14fd048844f57d655bef18af8bb0e5"} Jul 11 08:15:14.479032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:14.479005 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:14.555931 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:14.555903 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca\") pod \"666db48b-931c-44a9-80a4-b6f05b96ecbc\" (UID: \"666db48b-931c-44a9-80a4-b6f05b96ecbc\") " Jul 11 08:15:14.556237 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:14.556217 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "666db48b-931c-44a9-80a4-b6f05b96ecbc" (UID: "666db48b-931c-44a9-80a4-b6f05b96ecbc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:15:14.656564 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:14.656493 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/666db48b-931c-44a9-80a4-b6f05b96ecbc-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:15:15.355612 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:15.355585 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" Jul 11 08:15:15.355810 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:15.355628 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m" event={"ID":"666db48b-931c-44a9-80a4-b6f05b96ecbc","Type":"ContainerDied","Data":"6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b"} Jul 11 08:15:15.355810 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:15.355688 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6486b8711794e49f56224a2055ceab048f32bf6add9d45210f37fc56316fc99b" Jul 11 08:15:21.699663 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:21.699629 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk"] Jul 11 08:15:21.702616 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:21.702588 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-683f5ca7-29729295-crwnk"] Jul 11 08:15:22.710475 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:22.710439 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cd9c9de-b961-41eb-ad7d-3abf74f5206b" path="/var/lib/kubelet/pods/3cd9c9de-b961-41eb-ad7d-3abf74f5206b/volumes" Jul 11 08:15:25.381949 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:25.381915 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:15:30.869603 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:15:30.869572 2572 scope.go:117] "RemoveContainer" containerID="4087ad2dbdd5b9f5fb41d8f4239b4031a85a68a715e18f9d514d2ee355ae6096" Jul 11 08:16:22.193758 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:16:22.193720 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:16:36.087125 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:16:36.087089 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:17:20.401500 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.401468 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:17:20.401983 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.401898 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="666db48b-931c-44a9-80a4-b6f05b96ecbc" containerName="revoke-keys" Jul 11 08:17:20.401983 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.401911 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="666db48b-931c-44a9-80a4-b6f05b96ecbc" containerName="revoke-keys" Jul 11 08:17:20.401983 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.401973 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="666db48b-931c-44a9-80a4-b6f05b96ecbc" containerName="revoke-keys" Jul 11 08:17:20.405131 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.405109 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.407823 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.407797 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-59f9af-openshift-default-dockercfg-qv5kz\"" Jul 11 08:17:20.418162 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.418137 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:17:20.549433 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549397 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549433 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549436 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549646 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549646 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549580 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549646 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549631 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549781 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549668 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549781 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549690 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549781 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549730 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.549781 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.549772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czqm\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650612 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650612 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650633 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czqm\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650747 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.650843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651089 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651089 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.650927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651089 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.651046 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651297 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.651274 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651352 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.651331 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651400 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.651381 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.651444 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.651398 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.653188 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.653132 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.653422 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.653402 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.658267 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.658238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.658379 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.658341 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czqm\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm\") pod \"e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.715649 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.715622 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:20.856813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.856768 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:17:20.857066 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:17:20.857044 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2acb22_4be6_4574_a872_8f935a343714.slice/crio-34841a66b303f138e9342a9885d0a2c5a771ae13def56b732a6e7a5648445ccb WatchSource:0}: Error finding container 34841a66b303f138e9342a9885d0a2c5a771ae13def56b732a6e7a5648445ccb: Status 404 returned error can't find the container with id 34841a66b303f138e9342a9885d0a2c5a771ae13def56b732a6e7a5648445ccb Jul 11 08:17:20.859419 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.859381 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:20.859520 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.859496 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:20.859584 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:20.859551 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:21.073224 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:21.073184 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerStarted","Data":"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7"} Jul 11 08:17:21.073438 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:21.073236 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerStarted","Data":"34841a66b303f138e9342a9885d0a2c5a771ae13def56b732a6e7a5648445ccb"} Jul 11 08:17:21.096318 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:21.096259 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podStartSLOduration=1.096238764 podStartE2EDuration="1.096238764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:17:21.093842734 +0000 UTC m=+2330.939831649" watchObservedRunningTime="2026-07-11 08:17:21.096238764 +0000 UTC m=+2330.942227658" Jul 11 08:17:21.716328 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:21.716291 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:21.720862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:21.720835 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:22.079449 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:22.079412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:22.080218 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:22.080188 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:22.081599 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:22.081581 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:17:23.275234 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.275206 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:17:23.278739 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.278720 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.286620 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.286295 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 11 08:17:23.287246 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.287225 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 11 08:17:23.287369 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.287258 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jhxtl\"" Jul 11 08:17:23.287369 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.287227 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-59f9af-serving-cert\"" Jul 11 08:17:23.292560 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.292534 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:17:23.378380 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.378348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.378604 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.378581 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.378657 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.378628 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhq8s\" (UniqueName: \"kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.480112 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.480075 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhq8s\" (UniqueName: \"kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.480286 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.480118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.480286 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.480202 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.480366 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:17:23.480284 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-serving-cert: secret "maas-api-e2e-inf-b-59f9af-serving-cert" not found Jul 11 08:17:23.480366 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:17:23.480332 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls podName:1752a38a-8c94-46f2-b4ef-974b8c476336 nodeName:}" failed. No retries permitted until 2026-07-11 08:17:23.980316386 +0000 UTC m=+2333.826305257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls") pod "maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" (UID: "1752a38a-8c94-46f2-b4ef-974b8c476336") : secret "maas-api-e2e-inf-b-59f9af-serving-cert" not found Jul 11 08:17:23.480898 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.480874 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.495349 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.495309 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhq8s\" (UniqueName: \"kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.985862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.985828 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:23.988355 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:23.988330 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") pod \"maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:24.190514 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:24.190482 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:24.323629 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:24.323606 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:17:24.324817 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:17:24.324793 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1752a38a_8c94_46f2_b4ef_974b8c476336.slice/crio-77be6df20497fe899c34550962e8f7814833c2a2cd4b1b61416b93c3ad9dd6ce WatchSource:0}: Error finding container 77be6df20497fe899c34550962e8f7814833c2a2cd4b1b61416b93c3ad9dd6ce: Status 404 returned error can't find the container with id 77be6df20497fe899c34550962e8f7814833c2a2cd4b1b61416b93c3ad9dd6ce Jul 11 08:17:24.326715 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:24.326682 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 08:17:25.098220 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:25.098176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" event={"ID":"1752a38a-8c94-46f2-b4ef-974b8c476336","Type":"ContainerStarted","Data":"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2"} Jul 11 08:17:25.098401 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:25.098273 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" event={"ID":"1752a38a-8c94-46f2-b4ef-974b8c476336","Type":"ContainerStarted","Data":"77be6df20497fe899c34550962e8f7814833c2a2cd4b1b61416b93c3ad9dd6ce"} Jul 11 08:17:25.098401 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:25.098301 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:25.117131 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:25.117078 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" podStartSLOduration=1.733687442 podStartE2EDuration="2.11706301s" totalImagesPullingTime="383.375568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:17:25.116691079 +0000 UTC m=+2334.962679971" watchObservedRunningTime="2026-07-11 08:17:25.11706301 +0000 UTC m=+2334.963051929" Jul 11 08:17:30.899427 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:30.899392 2572 scope.go:117] "RemoveContainer" containerID="e14b491b88e76caaea3ecf00e029ccb9a409ce2be24ec8fcd17db4228c2501e0" Jul 11 08:17:31.111457 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:31.111426 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:17:34.085282 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:34.085246 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:17:36.173582 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.173505 2572 generic.go:356] "Generic (PLEG): container finished" podID="5c2acb22-4be6-4574-a872-8f935a343714" containerID="f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7" exitCode=0 Jul 11 08:17:36.173991 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.173576 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerDied","Data":"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7"} Jul 11 08:17:36.174070 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.174053 2572 scope.go:117] "RemoveContainer" containerID="f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7" Jul 11 08:17:36.175046 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.175018 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:36.175097 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.175083 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:36.175138 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:36.175111 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:17:37.080034 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:37.079990 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:37.181174 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:37.181141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerStarted","Data":"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3"} Jul 11 08:17:37.716174 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:37.716136 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:37.717226 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:37.717200 2572 patch_prober.go:28] interesting pod/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 11 08:17:37.717355 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:37.717261 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 11 08:17:38.716394 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:38.716364 2572 patch_prober.go:28] interesting pod/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 11 08:17:38.716811 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:38.716417 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 11 08:17:39.716061 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:39.716027 2572 patch_prober.go:28] interesting pod/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 11 08:17:39.716232 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:39.716086 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 11 08:17:40.716776 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:40.716744 2572 patch_prober.go:28] interesting pod/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 11 08:17:40.717246 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:40.716792 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 11 08:17:41.719781 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:41.719745 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:41.720163 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:41.720003 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:41.720736 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:41.720717 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:17:43.590619 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:17:43.590582 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:18:02.788499 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:02.788460 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:18:26.916379 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:26.916339 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm"] Jul 11 08:18:26.920085 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:26.920068 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:26.922827 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:26.922616 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2dw2n\"" Jul 11 08:18:26.926388 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:26.926365 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm"] Jul 11 08:18:26.977465 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:26.977436 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm\" (UID: \"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:27.078778 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:27.078746 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm\" (UID: \"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:27.079387 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:27.079359 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm\" (UID: \"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:27.232698 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:27.232611 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:27.368576 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:27.368531 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm"] Jul 11 08:18:27.368735 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:18:27.368594 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4961c3d_7f9b_44e9_bcd0_eaf6b7b7e076.slice/crio-cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42 WatchSource:0}: Error finding container cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42: Status 404 returned error can't find the container with id cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42 Jul 11 08:18:27.461787 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:27.461750 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" event={"ID":"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076","Type":"ContainerStarted","Data":"cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42"} Jul 11 08:18:28.475528 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:28.475433 2572 generic.go:356] "Generic (PLEG): container finished" podID="a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076" containerID="950a449b006c6c310422fc6317689f202a73f64b383124da1f1e939610e24380" exitCode=0 Jul 11 08:18:28.475528 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:28.475481 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" event={"ID":"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076","Type":"ContainerDied","Data":"950a449b006c6c310422fc6317689f202a73f64b383124da1f1e939610e24380"} Jul 11 08:18:29.616769 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:29.616742 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:29.703122 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:29.703092 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca\") pod \"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076\" (UID: \"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076\") " Jul 11 08:18:29.703422 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:29.703401 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076" (UID: "a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:18:29.804358 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:29.804310 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:18:30.489730 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.489676 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" Jul 11 08:18:30.489917 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.489678 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm" event={"ID":"a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076","Type":"ContainerDied","Data":"cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42"} Jul 11 08:18:30.489917 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.489807 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb27c413bb4cde937e7fbbfce08a22f62d65ae0358341a482de5a0119ae63c42" Jul 11 08:18:30.687303 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.687265 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:18:30.814028 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.813961 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:18:30.821072 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:30.821046 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:18:47.989236 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:18:47.989202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:19:03.427734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.427622 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:19:03.428192 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.427967 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" podUID="1752a38a-8c94-46f2-b4ef-974b8c476336" containerName="maas-api" containerID="cri-o://a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2" gracePeriod=30 Jul 11 08:19:03.883139 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.883116 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:19:03.911728 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.911690 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca\") pod \"1752a38a-8c94-46f2-b4ef-974b8c476336\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " Jul 11 08:19:03.911874 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.911774 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") pod \"1752a38a-8c94-46f2-b4ef-974b8c476336\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " Jul 11 08:19:03.911874 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.911817 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhq8s\" (UniqueName: \"kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s\") pod \"1752a38a-8c94-46f2-b4ef-974b8c476336\" (UID: \"1752a38a-8c94-46f2-b4ef-974b8c476336\") " Jul 11 08:19:03.912005 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.911981 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1752a38a-8c94-46f2-b4ef-974b8c476336" (UID: "1752a38a-8c94-46f2-b4ef-974b8c476336"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:19:03.912078 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.912063 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1752a38a-8c94-46f2-b4ef-974b8c476336-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:03.913755 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.913695 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1752a38a-8c94-46f2-b4ef-974b8c476336" (UID: "1752a38a-8c94-46f2-b4ef-974b8c476336"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 08:19:03.913870 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:03.913845 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s" (OuterVolumeSpecName: "kube-api-access-rhq8s") pod "1752a38a-8c94-46f2-b4ef-974b8c476336" (UID: "1752a38a-8c94-46f2-b4ef-974b8c476336"). InnerVolumeSpecName "kube-api-access-rhq8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:19:04.012727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.012680 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1752a38a-8c94-46f2-b4ef-974b8c476336-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:04.012727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.012727 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhq8s\" (UniqueName: \"kubernetes.io/projected/1752a38a-8c94-46f2-b4ef-974b8c476336-kube-api-access-rhq8s\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:04.687886 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.687850 2572 generic.go:356] "Generic (PLEG): container finished" podID="1752a38a-8c94-46f2-b4ef-974b8c476336" containerID="a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2" exitCode=0 Jul 11 08:19:04.687886 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.687889 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" Jul 11 08:19:04.688292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.687934 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" event={"ID":"1752a38a-8c94-46f2-b4ef-974b8c476336","Type":"ContainerDied","Data":"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2"} Jul 11 08:19:04.688292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.687989 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj" event={"ID":"1752a38a-8c94-46f2-b4ef-974b8c476336","Type":"ContainerDied","Data":"77be6df20497fe899c34550962e8f7814833c2a2cd4b1b61416b93c3ad9dd6ce"} Jul 11 08:19:04.688292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.688013 2572 scope.go:117] "RemoveContainer" containerID="a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2" Jul 11 08:19:04.697985 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.697970 2572 scope.go:117] "RemoveContainer" containerID="a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2" Jul 11 08:19:04.698254 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:19:04.698236 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2\": container with ID starting with a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2 not found: ID does not exist" containerID="a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2" Jul 11 08:19:04.698304 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.698263 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2"} err="failed to get container status \"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2\": rpc error: code = NotFound desc = could not find container \"a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2\": container with ID starting with a19b5c55865f1ffe70792e9ad2caff1c29870325464ae9c78040d956983b1ff2 not found: ID does not exist" Jul 11 08:19:04.711441 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.711421 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:19:04.716064 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:04.716046 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59f9af-6f555d86b7-d7qdj"] Jul 11 08:19:06.584542 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:06.584510 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:19:06.710544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:06.710514 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1752a38a-8c94-46f2-b4ef-974b8c476336" path="/var/lib/kubelet/pods/1752a38a-8c94-46f2-b4ef-974b8c476336/volumes" Jul 11 08:19:18.788841 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:18.788808 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:19:18.789453 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:18.789011 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" containerID="cri-o://28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3" gracePeriod=30 Jul 11 08:19:24.044310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.044274 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:19:24.098565 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098475 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.098565 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098518 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.098825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098576 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.098825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098615 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.098825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098636 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.098825 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098771 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.099032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098830 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.099032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098918 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.099032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098941 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4czqm\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm\") pod \"5c2acb22-4be6-4574-a872-8f935a343714\" (UID: \"5c2acb22-4be6-4574-a872-8f935a343714\") " Jul 11 08:19:24.099032 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.098973 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:19:24.099247 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099207 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:19:24.099316 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099241 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:19:24.099418 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099370 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:19:24.099418 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099398 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.099418 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099418 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2acb22-4be6-4574-a872-8f935a343714-istiod-ca-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.099799 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.099433 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:19:24.100905 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.100884 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:19:24.100995 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.100932 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:19:24.101330 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.101310 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm" (OuterVolumeSpecName: "kube-api-access-4czqm") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "kube-api-access-4czqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:19:24.101380 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.101310 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c2acb22-4be6-4574-a872-8f935a343714" (UID: "5c2acb22-4be6-4574-a872-8f935a343714"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 11 08:19:24.200263 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200233 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-envoy\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200263 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200259 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-istio-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200263 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200267 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-istio-data\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200276 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2acb22-4be6-4574-a872-8f935a343714-istio-podinfo\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200284 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-credential-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200294 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2acb22-4be6-4574-a872-8f935a343714-workload-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.200509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.200304 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4czqm\" (UniqueName: \"kubernetes.io/projected/5c2acb22-4be6-4574-a872-8f935a343714-kube-api-access-4czqm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:19:24.817671 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.817640 2572 generic.go:356] "Generic (PLEG): container finished" podID="5c2acb22-4be6-4574-a872-8f935a343714" containerID="28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3" exitCode=0 Jul 11 08:19:24.817884 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.817689 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" Jul 11 08:19:24.817884 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.817727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerDied","Data":"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3"} Jul 11 08:19:24.817884 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.817771 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2" event={"ID":"5c2acb22-4be6-4574-a872-8f935a343714","Type":"ContainerDied","Data":"34841a66b303f138e9342a9885d0a2c5a771ae13def56b732a6e7a5648445ccb"} Jul 11 08:19:24.817884 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.817792 2572 scope.go:117] "RemoveContainer" containerID="28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3" Jul 11 08:19:24.833321 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.833300 2572 scope.go:117] "RemoveContainer" containerID="f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7" Jul 11 08:19:24.842921 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.842792 2572 scope.go:117] "RemoveContainer" containerID="28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3" Jul 11 08:19:24.843143 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:19:24.843122 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3\": container with ID starting with 28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3 not found: ID does not exist" containerID="28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3" Jul 11 08:19:24.843227 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.843177 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3"} err="failed to get container status \"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3\": rpc error: code = NotFound desc = could not find container \"28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3\": container with ID starting with 28e311187df885b3dd10e944cc534d74c05ce0f299dacfb0b6ea3b727d2e37f3 not found: ID does not exist" Jul 11 08:19:24.843227 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.843197 2572 scope.go:117] "RemoveContainer" containerID="f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7" Jul 11 08:19:24.843431 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.843412 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:19:24.843555 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:19:24.843461 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7\": container with ID starting with f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7 not found: ID does not exist" containerID="f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7" Jul 11 08:19:24.843555 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.843496 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7"} err="failed to get container status \"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7\": rpc error: code = NotFound desc = could not find container \"f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7\": container with ID starting with f86e08b43b397e5ff8000dc2a2962f5d70021c6f6b0547c19abbd400adcf15e7 not found: ID does not exist" Jul 11 08:19:24.846097 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:24.846072 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-59f9af-openshift-default-6b9c9b57d7-hrtg2"] Jul 11 08:19:26.714165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:26.714131 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c2acb22-4be6-4574-a872-8f935a343714" path="/var/lib/kubelet/pods/5c2acb22-4be6-4574-a872-8f935a343714/volumes" Jul 11 08:19:27.080196 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:27.080163 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:19:28.284166 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284133 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284555 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284572 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284593 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1752a38a-8c94-46f2-b4ef-974b8c476336" containerName="maas-api" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284603 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1752a38a-8c94-46f2-b4ef-974b8c476336" containerName="maas-api" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284623 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076" containerName="revoke-keys" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284631 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076" containerName="revoke-keys" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284643 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:19:28.284664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284649 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:19:28.285090 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284744 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1752a38a-8c94-46f2-b4ef-974b8c476336" containerName="maas-api" Jul 11 08:19:28.285090 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284756 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:19:28.285090 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.284766 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076" containerName="revoke-keys" Jul 11 08:19:28.289658 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.289631 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.292240 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.292220 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f92f11-openshift-default-dockercfg-9bwbj\"" Jul 11 08:19:28.299542 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.299511 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:19:28.439749 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439714 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.439749 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439751 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56nbl\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.439974 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439820 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.439974 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439892 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.439974 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439941 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.440118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.439976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.440118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.440007 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.440118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.440091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.440244 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.440130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.540934 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.540847 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.540934 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.540906 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541020 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541160 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541426 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56nbl\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541426 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541426 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541304 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541426 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541382 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541579 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541442 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541579 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.541740 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.541693 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.543559 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.543531 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.543696 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.543678 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.548547 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.548522 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.549114 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.549095 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56nbl\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl\") pod \"e2e-api-b-f92f11-openshift-default-88855785f-pwgpj\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.601997 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.601955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:28.743595 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.743566 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:19:28.744698 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:19:28.744674 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6cee2fb_5d9a_478f_9ef2_36c0e1492a3d.slice/crio-2ac944314e769688bff9fe7dd09ab6ad31d278a792ffd23817f0dab3e1292bce WatchSource:0}: Error finding container 2ac944314e769688bff9fe7dd09ab6ad31d278a792ffd23817f0dab3e1292bce: Status 404 returned error can't find the container with id 2ac944314e769688bff9fe7dd09ab6ad31d278a792ffd23817f0dab3e1292bce Jul 11 08:19:28.747000 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.746969 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:28.747072 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.747033 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:28.747072 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.747062 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:28.846085 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.846034 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerStarted","Data":"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17"} Jul 11 08:19:28.846269 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.846117 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerStarted","Data":"2ac944314e769688bff9fe7dd09ab6ad31d278a792ffd23817f0dab3e1292bce"} Jul 11 08:19:28.868082 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:28.868014 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podStartSLOduration=0.867979698 podStartE2EDuration="867.979698ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:19:28.865272892 +0000 UTC m=+2458.711261784" watchObservedRunningTime="2026-07-11 08:19:28.867979698 +0000 UTC m=+2458.713968593" Jul 11 08:19:29.603174 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:29.603136 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:29.608030 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:29.608004 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:29.852292 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:29.852262 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:29.853433 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:29.853366 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:38.483438 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:38.483394 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:19:45.893898 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:45.893853 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 11 08:19:45.894334 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:45.894031 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 11 08:19:46.955896 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.955860 2572 generic.go:356] "Generic (PLEG): container finished" podID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerID="113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17" exitCode=0 Jul 11 08:19:46.956296 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.955933 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerDied","Data":"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17"} Jul 11 08:19:46.956338 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.956303 2572 scope.go:117] "RemoveContainer" containerID="113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17" Jul 11 08:19:46.957306 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.957275 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:46.957380 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.957347 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:46.957421 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:46.957390 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:19:47.964348 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:47.964311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerStarted","Data":"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6"} Jul 11 08:19:48.602798 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:48.602750 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:48.603085 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:48.603061 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Jul 11 08:19:48.603142 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:48.603123 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Jul 11 08:19:49.602992 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:49.602956 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Jul 11 08:19:49.603468 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:49.603007 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Jul 11 08:19:50.602768 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:50.602729 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Jul 11 08:19:50.602952 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:50.602783 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Jul 11 08:19:51.603192 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:51.603155 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Jul 11 08:19:51.603581 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:51.603205 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Jul 11 08:19:52.605480 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:52.605450 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:52.605908 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:52.605885 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:52.606543 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:52.606525 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:19:58.980198 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:19:58.980167 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:20:23.184345 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:23.184265 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:20:35.248839 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:35.248804 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:20:35.249279 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:35.249040 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" containerID="cri-o://b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6" gracePeriod=30 Jul 11 08:20:37.608280 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:37.608243 2572 patch_prober.go:28] interesting pod/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 11 08:20:37.608669 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:37.608317 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 08:20:40.508648 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.508582 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:20:40.603973 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.603936 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604177 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.603985 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604177 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604014 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604177 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604100 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604177 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604124 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604177 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604156 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604429 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604193 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56nbl\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604429 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604238 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604429 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604225 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:20:40.604429 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604267 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token\") pod \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\" (UID: \"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d\") " Jul 11 08:20:40.604638 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604449 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:20:40.604638 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604537 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:20:40.604638 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604559 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:20:40.604828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604639 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.604828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604659 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-credential-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.604828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604675 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-workload-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.605014 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.604991 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:20:40.606459 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.606433 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:20:40.606575 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.606533 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl" (OuterVolumeSpecName: "kube-api-access-56nbl") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "kube-api-access-56nbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:20:40.606626 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.606614 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:20:40.606626 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.606611 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" (UID: "b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706079 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706116 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-podinfo\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706132 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-data\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706144 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istiod-ca-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706161 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-istio-envoy\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:40.713167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:40.706173 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56nbl\" (UniqueName: \"kubernetes.io/projected/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d-kube-api-access-56nbl\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:20:41.273403 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.273370 2572 generic.go:356] "Generic (PLEG): container finished" podID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerID="b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6" exitCode=0 Jul 11 08:20:41.273572 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.273430 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" Jul 11 08:20:41.273572 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.273455 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerDied","Data":"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6"} Jul 11 08:20:41.273572 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.273512 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj" event={"ID":"b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d","Type":"ContainerDied","Data":"2ac944314e769688bff9fe7dd09ab6ad31d278a792ffd23817f0dab3e1292bce"} Jul 11 08:20:41.273572 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.273536 2572 scope.go:117] "RemoveContainer" containerID="b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6" Jul 11 08:20:41.282743 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.282728 2572 scope.go:117] "RemoveContainer" containerID="113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17" Jul 11 08:20:41.293831 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.293756 2572 scope.go:117] "RemoveContainer" containerID="b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6" Jul 11 08:20:41.294102 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:20:41.294068 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6\": container with ID starting with b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6 not found: ID does not exist" containerID="b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6" Jul 11 08:20:41.294167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.294105 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6"} err="failed to get container status \"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6\": rpc error: code = NotFound desc = could not find container \"b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6\": container with ID starting with b8a0e27bf690608c884d54ccc80d9a82931b4e4d50c517c85d9952a85a271ad6 not found: ID does not exist" Jul 11 08:20:41.294167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.294128 2572 scope.go:117] "RemoveContainer" containerID="113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17" Jul 11 08:20:41.294420 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:20:41.294404 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17\": container with ID starting with 113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17 not found: ID does not exist" containerID="113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17" Jul 11 08:20:41.294471 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.294424 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17"} err="failed to get container status \"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17\": rpc error: code = NotFound desc = could not find container \"113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17\": container with ID starting with 113803a1c90780ddcd3bac47d3f3f83117f0e18f3e77c2f761112f3d5632df17 not found: ID does not exist" Jul 11 08:20:41.295132 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.295118 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:20:41.297534 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:41.297516 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f92f11-openshift-default-88855785f-pwgpj"] Jul 11 08:20:42.711251 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:42.711218 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" path="/var/lib/kubelet/pods/b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d/volumes" Jul 11 08:20:44.190727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.190674 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:20:44.375774 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.375737 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:20:44.376324 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376304 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.376367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376329 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.376367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376357 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.376367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376365 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.376493 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376481 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2acb22-4be6-4574-a872-8f935a343714" containerName="istio-proxy" Jul 11 08:20:44.376529 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376498 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.376529 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.376508 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6cee2fb-5d9a-478f-9ef2-36c0e1492a3d" containerName="istio-proxy" Jul 11 08:20:44.381389 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.381365 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.385086 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.385063 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-12fff0-openshift-default-dockercfg-7mpcs\"" Jul 11 08:20:44.395306 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.395268 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:20:44.432199 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432383 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432211 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69pc9\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432383 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432383 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432336 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432383 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432363 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432617 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432437 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432617 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432478 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432617 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.432617 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.432590 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533464 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533429 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533464 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533504 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533648 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.533792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533669 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69pc9\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534122 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533968 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534122 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.533973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534122 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.534027 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534284 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.534189 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534372 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.534349 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.534525 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.534506 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.535963 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.535937 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.536201 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.536179 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.542110 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.542089 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.542499 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.542480 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69pc9\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9\") pod \"e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.690941 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.690908 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:44.831970 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.831934 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:20:44.832371 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:20:44.832345 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cef1d4_0602_48ea_8eb7_75ba7d6e75d5.slice/crio-8b75f4bbd35960a5b03bfc78c6190dbf6389cc10fa55d1ec85b19397954439a4 WatchSource:0}: Error finding container 8b75f4bbd35960a5b03bfc78c6190dbf6389cc10fa55d1ec85b19397954439a4: Status 404 returned error can't find the container with id 8b75f4bbd35960a5b03bfc78c6190dbf6389cc10fa55d1ec85b19397954439a4 Jul 11 08:20:44.834565 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.834532 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:20:44.834679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.834625 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:20:44.834679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:44.834658 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:20:45.298912 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:45.298873 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerStarted","Data":"339ba70f601d9737c00a0774400aed27616034cc5a6246093a0145f1f9f6e928"} Jul 11 08:20:45.299291 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:45.298930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerStarted","Data":"8b75f4bbd35960a5b03bfc78c6190dbf6389cc10fa55d1ec85b19397954439a4"} Jul 11 08:20:45.320206 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:45.320154 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podStartSLOduration=1.320135904 podStartE2EDuration="1.320135904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:20:45.318415861 +0000 UTC m=+2535.164404768" watchObservedRunningTime="2026-07-11 08:20:45.320135904 +0000 UTC m=+2535.166124797" Jul 11 08:20:45.692112 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:45.692009 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:45.697051 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:45.697024 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:46.305222 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:46.305188 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:46.306300 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:46.306278 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:20:53.086269 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:20:53.086237 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:21:01.306466 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.306428 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Jul 11 08:21:01.306915 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.306495 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Jul 11 08:21:01.397083 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.397047 2572 generic.go:356] "Generic (PLEG): container finished" podID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerID="339ba70f601d9737c00a0774400aed27616034cc5a6246093a0145f1f9f6e928" exitCode=0 Jul 11 08:21:01.397280 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.397120 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerDied","Data":"339ba70f601d9737c00a0774400aed27616034cc5a6246093a0145f1f9f6e928"} Jul 11 08:21:01.397559 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.397542 2572 scope.go:117] "RemoveContainer" containerID="339ba70f601d9737c00a0774400aed27616034cc5a6246093a0145f1f9f6e928" Jul 11 08:21:01.398509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.398479 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:21:01.398597 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.398544 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:21:01.398597 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:01.398573 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 11 08:21:02.405392 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:02.405359 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerStarted","Data":"42c211f0d7012ae11872c2c83a197b74477dbb19577acffeaa6d5c5a35c96d12"} Jul 11 08:21:02.691400 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:02.691302 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:21:02.691620 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:02.691597 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Jul 11 08:21:02.691678 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:02.691657 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Jul 11 08:21:03.692106 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:03.692064 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Jul 11 08:21:03.692491 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:03.692124 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Jul 11 08:21:04.691355 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:04.691323 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Jul 11 08:21:04.691544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:04.691375 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Jul 11 08:21:05.691656 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:05.691621 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Jul 11 08:21:05.692074 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:05.691738 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Jul 11 08:21:06.695930 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:06.695898 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:21:06.696311 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:06.696241 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:21:06.697099 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:06.697073 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:21:08.877901 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:08.877858 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:21:29.088530 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:29.088493 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:21:30.958664 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:30.958633 2572 scope.go:117] "RemoveContainer" containerID="2fb13d9f50fe9d2fb7b7fe9fb71ce0d37255c271837e8ed27730c912678b7c40" Jul 11 08:21:45.385348 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:21:45.385305 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:22:00.483876 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:22:00.483800 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:22:17.378792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:22:17.378754 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:22:30.790115 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:22:30.790086 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:22:48.091233 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:22:48.091199 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:23:05.382644 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:05.382606 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:23:23.287843 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:23.287767 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:23:30.853092 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:30.853059 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:23:30.863154 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:30.863132 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:23:38.388398 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:38.388368 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:23:53.686524 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:23:53.686485 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:24:08.589828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:08.589794 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:24:28.759095 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.759063 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:24:28.762670 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.762654 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.766575 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.766544 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"kube-root-ca.crt\"" Jul 11 08:24:28.768007 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.767986 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"rate-test-model-f178a6-kserve-self-signed-certs\"" Jul 11 08:24:28.768133 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.768010 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"openshift-service-ca.crt\"" Jul 11 08:24:28.768133 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.768035 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f178a6\"/\"default-dockercfg-chfh6\"" Jul 11 08:24:28.774498 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.774473 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:24:28.853106 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.853264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853109 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.853264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.853264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853214 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.853264 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853250 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.853401 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.853274 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq2nx\" (UniqueName: \"kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954509 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954511 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954846 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954846 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954581 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954846 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.954846 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.954640 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq2nx\" (UniqueName: \"kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.955069 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.955019 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.955069 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.955028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.955069 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.955061 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.956770 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.956750 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.957027 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.957009 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:28.962526 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:28.962499 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq2nx\" (UniqueName: \"kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx\") pod \"rate-test-model-f178a6-kserve-69c8854c5f-mxxsp\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:29.072792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:29.072688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:29.204226 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:29.204197 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:24:29.205043 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:24:29.205019 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1c9d382_c26a_463a_a0c2_7871c738697f.slice/crio-a8588b134eeaed5c821f0ff35a445fead7d4d957f0fa27c526298fc32cde67f5 WatchSource:0}: Error finding container a8588b134eeaed5c821f0ff35a445fead7d4d957f0fa27c526298fc32cde67f5: Status 404 returned error can't find the container with id a8588b134eeaed5c821f0ff35a445fead7d4d957f0fa27c526298fc32cde67f5 Jul 11 08:24:29.207014 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:29.206996 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 08:24:29.483385 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:29.483299 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:24:29.613522 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:29.613480 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" event={"ID":"f1c9d382-c26a-463a-a0c2-7871c738697f","Type":"ContainerStarted","Data":"a8588b134eeaed5c821f0ff35a445fead7d4d957f0fa27c526298fc32cde67f5"} Jul 11 08:24:30.620877 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:30.620782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" event={"ID":"f1c9d382-c26a-463a-a0c2-7871c738697f","Type":"ContainerStarted","Data":"b317c0f6174a87be9df679a9efee5a4568d4084422db11e2ccb1ad604c74e860"} Jul 11 08:24:30.639354 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:30.639304 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" podStartSLOduration=1.550702159 podStartE2EDuration="2.639289199s" totalImagesPullingTime="1.08858704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:24:30.636223484 +0000 UTC m=+2760.482212377" watchObservedRunningTime="2026-07-11 08:24:30.639289199 +0000 UTC m=+2760.485278090" Jul 11 08:24:39.073885 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:39.073841 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:39.074305 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:39.073937 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:39.089990 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:39.089959 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:39.686782 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:39.686756 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:24:44.285968 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:24:44.285932 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:25:00.180258 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:00.180174 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:25:13.783975 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:13.783940 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:25:32.979227 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:32.979191 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:25:39.927491 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:39.927450 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:25:41.629745 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.629692 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863440953904"] Jul 11 08:25:41.633180 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.633158 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:41.635774 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.635749 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-lwq6f\"" Jul 11 08:25:41.638925 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.638896 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863440953904"] Jul 11 08:25:41.722822 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.722789 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drcrg\" (UniqueName: \"kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg\") pod \"test-curl-7591-139863440953904\" (UID: \"10ab4d5e-ee85-494d-88b6-90ae6e16094b\") " pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:41.823869 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.823840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drcrg\" (UniqueName: \"kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg\") pod \"test-curl-7591-139863440953904\" (UID: \"10ab4d5e-ee85-494d-88b6-90ae6e16094b\") " pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:41.831448 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.831420 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drcrg\" (UniqueName: \"kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg\") pod \"test-curl-7591-139863440953904\" (UID: \"10ab4d5e-ee85-494d-88b6-90ae6e16094b\") " pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:41.945266 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:41.945194 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:42.079249 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:42.079223 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863440953904"] Jul 11 08:25:42.083944 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:42.083903 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10ab4d5e_ee85_494d_88b6_90ae6e16094b.slice/crio-cb39809ffe0037b24a2d5973e5e9acfa777afdeadae48f049937f674d9d2563a WatchSource:0}: Error finding container cb39809ffe0037b24a2d5973e5e9acfa777afdeadae48f049937f674d9d2563a: Status 404 returned error can't find the container with id cb39809ffe0037b24a2d5973e5e9acfa777afdeadae48f049937f674d9d2563a Jul 11 08:25:43.054065 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:43.053977 2572 generic.go:356] "Generic (PLEG): container finished" podID="10ab4d5e-ee85-494d-88b6-90ae6e16094b" containerID="2e0adb6945ab969a4588c418eeeab70ffd2f8082733d11b4ad32a40ed1c5a617" exitCode=0 Jul 11 08:25:43.054065 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:43.054016 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863440953904" event={"ID":"10ab4d5e-ee85-494d-88b6-90ae6e16094b","Type":"ContainerDied","Data":"2e0adb6945ab969a4588c418eeeab70ffd2f8082733d11b4ad32a40ed1c5a617"} Jul 11 08:25:43.054065 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:43.054052 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863440953904" event={"ID":"10ab4d5e-ee85-494d-88b6-90ae6e16094b","Type":"ContainerStarted","Data":"cb39809ffe0037b24a2d5973e5e9acfa777afdeadae48f049937f674d9d2563a"} Jul 11 08:25:44.194116 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.194088 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:44.229138 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.229075 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863440953904_10ab4d5e-ee85-494d-88b6-90ae6e16094b/test-curl-7591-139863440953904/0.log" Jul 11 08:25:44.251749 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.251713 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863440953904"] Jul 11 08:25:44.255016 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.254995 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863440953904"] Jul 11 08:25:44.352845 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.352778 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drcrg\" (UniqueName: \"kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg\") pod \"10ab4d5e-ee85-494d-88b6-90ae6e16094b\" (UID: \"10ab4d5e-ee85-494d-88b6-90ae6e16094b\") " Jul 11 08:25:44.354817 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.354791 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg" (OuterVolumeSpecName: "kube-api-access-drcrg") pod "10ab4d5e-ee85-494d-88b6-90ae6e16094b" (UID: "10ab4d5e-ee85-494d-88b6-90ae6e16094b"). InnerVolumeSpecName "kube-api-access-drcrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:44.375615 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.375588 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863441913776"] Jul 11 08:25:44.375985 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.375973 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ab4d5e-ee85-494d-88b6-90ae6e16094b" containerName="test-curl-7591-139863440953904" Jul 11 08:25:44.376036 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.375987 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ab4d5e-ee85-494d-88b6-90ae6e16094b" containerName="test-curl-7591-139863440953904" Jul 11 08:25:44.376071 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.376065 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ab4d5e-ee85-494d-88b6-90ae6e16094b" containerName="test-curl-7591-139863440953904" Jul 11 08:25:44.379356 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.379341 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:44.384218 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.384196 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441913776"] Jul 11 08:25:44.453940 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.453912 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drcrg\" (UniqueName: \"kubernetes.io/projected/10ab4d5e-ee85-494d-88b6-90ae6e16094b-kube-api-access-drcrg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:44.554404 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.554368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmcx6\" (UniqueName: \"kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6\") pod \"test-curl-7591-139863441913776\" (UID: \"1e058df7-453c-40dd-a494-8b995409870b\") " pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:44.655137 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.655060 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmcx6\" (UniqueName: \"kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6\") pod \"test-curl-7591-139863441913776\" (UID: \"1e058df7-453c-40dd-a494-8b995409870b\") " pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:44.662713 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.662684 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmcx6\" (UniqueName: \"kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6\") pod \"test-curl-7591-139863441913776\" (UID: \"1e058df7-453c-40dd-a494-8b995409870b\") " pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:44.689747 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.689698 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:44.715880 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.715856 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10ab4d5e-ee85-494d-88b6-90ae6e16094b" path="/var/lib/kubelet/pods/10ab4d5e-ee85-494d-88b6-90ae6e16094b/volumes" Jul 11 08:25:44.823252 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:44.823223 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441913776"] Jul 11 08:25:44.824345 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:44.824321 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e058df7_453c_40dd_a494_8b995409870b.slice/crio-35b34f6433a77601e48b6d2b3902efef148b7be6bbdc50364c7581891ce0d6b3 WatchSource:0}: Error finding container 35b34f6433a77601e48b6d2b3902efef148b7be6bbdc50364c7581891ce0d6b3: Status 404 returned error can't find the container with id 35b34f6433a77601e48b6d2b3902efef148b7be6bbdc50364c7581891ce0d6b3 Jul 11 08:25:45.067274 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:45.067250 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863440953904" Jul 11 08:25:45.067463 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:45.067305 2572 scope.go:117] "RemoveContainer" containerID="2e0adb6945ab969a4588c418eeeab70ffd2f8082733d11b4ad32a40ed1c5a617" Jul 11 08:25:45.068610 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:45.068518 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441913776" event={"ID":"1e058df7-453c-40dd-a494-8b995409870b","Type":"ContainerStarted","Data":"35b34f6433a77601e48b6d2b3902efef148b7be6bbdc50364c7581891ce0d6b3"} Jul 11 08:25:46.075549 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:46.075518 2572 generic.go:356] "Generic (PLEG): container finished" podID="1e058df7-453c-40dd-a494-8b995409870b" containerID="c4557c580758e0aee00d19ab14197c55176a77a4a27d41c7c692d4675207a68e" exitCode=0 Jul 11 08:25:46.075978 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:46.075604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441913776" event={"ID":"1e058df7-453c-40dd-a494-8b995409870b","Type":"ContainerDied","Data":"c4557c580758e0aee00d19ab14197c55176a77a4a27d41c7c692d4675207a68e"} Jul 11 08:25:46.786873 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:46.786837 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:25:47.230807 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.230775 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:47.249946 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.249919 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863441913776_1e058df7-453c-40dd-a494-8b995409870b/test-curl-7591-139863441913776/0.log" Jul 11 08:25:47.272919 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.272891 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863441913776"] Jul 11 08:25:47.277983 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.277956 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863441913776"] Jul 11 08:25:47.382666 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.382583 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmcx6\" (UniqueName: \"kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6\") pod \"1e058df7-453c-40dd-a494-8b995409870b\" (UID: \"1e058df7-453c-40dd-a494-8b995409870b\") " Jul 11 08:25:47.384630 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.384603 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6" (OuterVolumeSpecName: "kube-api-access-lmcx6") pod "1e058df7-453c-40dd-a494-8b995409870b" (UID: "1e058df7-453c-40dd-a494-8b995409870b"). InnerVolumeSpecName "kube-api-access-lmcx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:47.434464 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.434430 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:47.435065 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.435047 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e058df7-453c-40dd-a494-8b995409870b" containerName="test-curl-7591-139863441913776" Jul 11 08:25:47.435127 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.435069 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e058df7-453c-40dd-a494-8b995409870b" containerName="test-curl-7591-139863441913776" Jul 11 08:25:47.435190 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.435178 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e058df7-453c-40dd-a494-8b995409870b" containerName="test-curl-7591-139863441913776" Jul 11 08:25:47.445899 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.445870 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:47.449510 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.449479 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:47.484152 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.484126 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmcx6\" (UniqueName: \"kubernetes.io/projected/1e058df7-453c-40dd-a494-8b995409870b-kube-api-access-lmcx6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:47.585251 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.585213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s7f9\" (UniqueName: \"kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9\") pod \"test-curl-7591-139863441221360\" (UID: \"ec81c0bc-1654-41af-b1be-08bc4c546ce0\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:47.686793 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.686699 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s7f9\" (UniqueName: \"kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9\") pod \"test-curl-7591-139863441221360\" (UID: \"ec81c0bc-1654-41af-b1be-08bc4c546ce0\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:47.694343 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.694320 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s7f9\" (UniqueName: \"kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9\") pod \"test-curl-7591-139863441221360\" (UID: \"ec81c0bc-1654-41af-b1be-08bc4c546ce0\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:47.757374 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.757347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:47.887792 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:47.887763 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec81c0bc_1654_41af_b1be_08bc4c546ce0.slice/crio-fa56d0fa6f190c12b67cae7460b09fc09853140edd31e936b46c6f1de6fd2826 WatchSource:0}: Error finding container fa56d0fa6f190c12b67cae7460b09fc09853140edd31e936b46c6f1de6fd2826: Status 404 returned error can't find the container with id fa56d0fa6f190c12b67cae7460b09fc09853140edd31e936b46c6f1de6fd2826 Jul 11 08:25:47.887938 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:47.887843 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:48.088892 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:48.088857 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441221360" event={"ID":"ec81c0bc-1654-41af-b1be-08bc4c546ce0","Type":"ContainerStarted","Data":"fa56d0fa6f190c12b67cae7460b09fc09853140edd31e936b46c6f1de6fd2826"} Jul 11 08:25:48.090051 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:48.090031 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:48.090165 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:48.090041 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35b34f6433a77601e48b6d2b3902efef148b7be6bbdc50364c7581891ce0d6b3" Jul 11 08:25:48.100067 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:25:48.100045 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7591-139863441913776\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="1e058df7-453c-40dd-a494-8b995409870b" pod="openshift-ingress/test-curl-7591-139863441913776" Jul 11 08:25:48.711379 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:48.711345 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e058df7-453c-40dd-a494-8b995409870b" path="/var/lib/kubelet/pods/1e058df7-453c-40dd-a494-8b995409870b/volumes" Jul 11 08:25:49.096983 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:49.096945 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec81c0bc-1654-41af-b1be-08bc4c546ce0" containerID="d344b4130bddb62a78f97817b73feb3376ebdb7f5bd10b22f2ec48fe931f706b" exitCode=0 Jul 11 08:25:49.097173 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:49.097030 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441221360" event={"ID":"ec81c0bc-1654-41af-b1be-08bc4c546ce0","Type":"ContainerDied","Data":"d344b4130bddb62a78f97817b73feb3376ebdb7f5bd10b22f2ec48fe931f706b"} Jul 11 08:25:50.241448 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.241426 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.261002 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.260972 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863441221360_ec81c0bc-1654-41af-b1be-08bc4c546ce0/test-curl-7591-139863441221360/0.log" Jul 11 08:25:50.285336 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.285297 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:50.287887 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.287867 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:50.308004 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.307969 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s7f9\" (UniqueName: \"kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9\") pod \"ec81c0bc-1654-41af-b1be-08bc4c546ce0\" (UID: \"ec81c0bc-1654-41af-b1be-08bc4c546ce0\") " Jul 11 08:25:50.309960 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.309934 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9" (OuterVolumeSpecName: "kube-api-access-5s7f9") pod "ec81c0bc-1654-41af-b1be-08bc4c546ce0" (UID: "ec81c0bc-1654-41af-b1be-08bc4c546ce0"). InnerVolumeSpecName "kube-api-access-5s7f9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:50.402408 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.402327 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:50.402743 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.402730 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec81c0bc-1654-41af-b1be-08bc4c546ce0" containerName="test-curl-7591-139863441221360" Jul 11 08:25:50.402796 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.402744 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec81c0bc-1654-41af-b1be-08bc4c546ce0" containerName="test-curl-7591-139863441221360" Jul 11 08:25:50.402832 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.402814 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec81c0bc-1654-41af-b1be-08bc4c546ce0" containerName="test-curl-7591-139863441221360" Jul 11 08:25:50.406199 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.406024 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.408769 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.408737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qdvk\" (UniqueName: \"kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk\") pod \"test-curl-7591-139863441221360\" (UID: \"70d8ef51-dc53-4a72-8199-e9da22fd22fe\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.408909 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.408878 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s7f9\" (UniqueName: \"kubernetes.io/projected/ec81c0bc-1654-41af-b1be-08bc4c546ce0-kube-api-access-5s7f9\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:50.409973 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.409947 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:50.509943 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.509916 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qdvk\" (UniqueName: \"kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk\") pod \"test-curl-7591-139863441221360\" (UID: \"70d8ef51-dc53-4a72-8199-e9da22fd22fe\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.518093 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.518066 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qdvk\" (UniqueName: \"kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk\") pod \"test-curl-7591-139863441221360\" (UID: \"70d8ef51-dc53-4a72-8199-e9da22fd22fe\") " pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.712560 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.712482 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec81c0bc-1654-41af-b1be-08bc4c546ce0" path="/var/lib/kubelet/pods/ec81c0bc-1654-41af-b1be-08bc4c546ce0/volumes" Jul 11 08:25:50.715734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.715716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:50.846974 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:50.846951 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:50.848261 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:50.848223 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70d8ef51_dc53_4a72_8199_e9da22fd22fe.slice/crio-8585c42e24895a2372b2d754fcb85df6ea1956cfc8e192b670aa26f876153973 WatchSource:0}: Error finding container 8585c42e24895a2372b2d754fcb85df6ea1956cfc8e192b670aa26f876153973: Status 404 returned error can't find the container with id 8585c42e24895a2372b2d754fcb85df6ea1956cfc8e192b670aa26f876153973 Jul 11 08:25:51.109497 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:51.109463 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441221360" event={"ID":"70d8ef51-dc53-4a72-8199-e9da22fd22fe","Type":"ContainerStarted","Data":"8585c42e24895a2372b2d754fcb85df6ea1956cfc8e192b670aa26f876153973"} Jul 11 08:25:51.110788 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:51.110766 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:51.110905 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:51.110806 2572 scope.go:117] "RemoveContainer" containerID="d344b4130bddb62a78f97817b73feb3376ebdb7f5bd10b22f2ec48fe931f706b" Jul 11 08:25:52.118004 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:52.117966 2572 generic.go:356] "Generic (PLEG): container finished" podID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" containerID="f2d00191c9cab3f74d28e812af14fa2aa4cc54137113d309444129f492b537d6" exitCode=0 Jul 11 08:25:52.118382 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:52.118053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863441221360" event={"ID":"70d8ef51-dc53-4a72-8199-e9da22fd22fe","Type":"ContainerDied","Data":"f2d00191c9cab3f74d28e812af14fa2aa4cc54137113d309444129f492b537d6"} Jul 11 08:25:53.261556 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.261533 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:53.280698 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.280670 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863441221360_70d8ef51-dc53-4a72-8199-e9da22fd22fe/test-curl-7591-139863441221360/0.log" Jul 11 08:25:53.302542 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.302516 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:53.308985 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.308963 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863441221360"] Jul 11 08:25:53.335341 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.335318 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qdvk\" (UniqueName: \"kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk\") pod \"70d8ef51-dc53-4a72-8199-e9da22fd22fe\" (UID: \"70d8ef51-dc53-4a72-8199-e9da22fd22fe\") " Jul 11 08:25:53.337363 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.337334 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk" (OuterVolumeSpecName: "kube-api-access-6qdvk") pod "70d8ef51-dc53-4a72-8199-e9da22fd22fe" (UID: "70d8ef51-dc53-4a72-8199-e9da22fd22fe"). InnerVolumeSpecName "kube-api-access-6qdvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:53.420009 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.419948 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:53.420335 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.420323 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" containerName="test-curl-7591-139863441221360" Jul 11 08:25:53.420376 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.420336 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" containerName="test-curl-7591-139863441221360" Jul 11 08:25:53.420421 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.420400 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" containerName="test-curl-7591-139863441221360" Jul 11 08:25:53.423360 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.423346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:53.430035 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.430013 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:53.436005 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.435983 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhpd8\" (UniqueName: \"kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8\") pod \"test-curl-7591-139863442011456\" (UID: \"f8346767-9d72-450d-aabb-968da6179b26\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:53.436182 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.436161 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qdvk\" (UniqueName: \"kubernetes.io/projected/70d8ef51-dc53-4a72-8199-e9da22fd22fe-kube-api-access-6qdvk\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:53.537489 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.537438 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhpd8\" (UniqueName: \"kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8\") pod \"test-curl-7591-139863442011456\" (UID: \"f8346767-9d72-450d-aabb-968da6179b26\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:53.547638 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.547605 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhpd8\" (UniqueName: \"kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8\") pod \"test-curl-7591-139863442011456\" (UID: \"f8346767-9d72-450d-aabb-968da6179b26\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:53.733863 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.733775 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:53.861149 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:53.861117 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:53.862450 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:53.862429 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8346767_9d72_450d_aabb_968da6179b26.slice/crio-454123811abdcafb6fd6cc0157dc9f9091ced0526ba47d872ca2057085d3e98a WatchSource:0}: Error finding container 454123811abdcafb6fd6cc0157dc9f9091ced0526ba47d872ca2057085d3e98a: Status 404 returned error can't find the container with id 454123811abdcafb6fd6cc0157dc9f9091ced0526ba47d872ca2057085d3e98a Jul 11 08:25:54.132841 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:54.132802 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863442011456" event={"ID":"f8346767-9d72-450d-aabb-968da6179b26","Type":"ContainerStarted","Data":"454123811abdcafb6fd6cc0157dc9f9091ced0526ba47d872ca2057085d3e98a"} Jul 11 08:25:54.134148 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:54.134126 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:54.134293 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:54.134125 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8585c42e24895a2372b2d754fcb85df6ea1956cfc8e192b670aa26f876153973" Jul 11 08:25:54.144965 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:25:54.144934 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7591-139863441221360\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" pod="openshift-ingress/test-curl-7591-139863441221360" Jul 11 08:25:54.711805 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:54.711773 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70d8ef51-dc53-4a72-8199-e9da22fd22fe" path="/var/lib/kubelet/pods/70d8ef51-dc53-4a72-8199-e9da22fd22fe/volumes" Jul 11 08:25:54.947850 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:54.947809 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" podUID="f1c9d382-c26a-463a-a0c2-7871c738697f" containerName="main" containerID="cri-o://b317c0f6174a87be9df679a9efee5a4568d4084422db11e2ccb1ad604c74e860" gracePeriod=45 Jul 11 08:25:55.144386 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.144357 2572 generic.go:356] "Generic (PLEG): container finished" podID="f1c9d382-c26a-463a-a0c2-7871c738697f" containerID="b317c0f6174a87be9df679a9efee5a4568d4084422db11e2ccb1ad604c74e860" exitCode=0 Jul 11 08:25:55.144514 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.144426 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" event={"ID":"f1c9d382-c26a-463a-a0c2-7871c738697f","Type":"ContainerDied","Data":"b317c0f6174a87be9df679a9efee5a4568d4084422db11e2ccb1ad604c74e860"} Jul 11 08:25:55.150523 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.150494 2572 generic.go:356] "Generic (PLEG): container finished" podID="f8346767-9d72-450d-aabb-968da6179b26" containerID="11cbf18152960361deff2ad07debeea6fabc10b4de8bdaf393721e1c45b38530" exitCode=0 Jul 11 08:25:55.150665 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.150536 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863442011456" event={"ID":"f8346767-9d72-450d-aabb-968da6179b26","Type":"ContainerDied","Data":"11cbf18152960361deff2ad07debeea6fabc10b4de8bdaf393721e1c45b38530"} Jul 11 08:25:55.198403 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.198379 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:25:55.253838 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.253801 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.253850 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.253917 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254001 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.253994 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254163 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254021 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254163 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254074 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq2nx\" (UniqueName: \"kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx\") pod \"f1c9d382-c26a-463a-a0c2-7871c738697f\" (UID: \"f1c9d382-c26a-463a-a0c2-7871c738697f\") " Jul 11 08:25:55.254163 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254113 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:25:55.254310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254238 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:25:55.254310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254288 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home" (OuterVolumeSpecName: "home") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:25:55.254406 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254394 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-model-cache\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:55.254455 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254413 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-tmp-dir\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:55.254455 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.254426 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-home\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:55.256064 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.256035 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 08:25:55.256203 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.256187 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm" (OuterVolumeSpecName: "dshm") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:25:55.256520 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.256497 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx" (OuterVolumeSpecName: "kube-api-access-qq2nx") pod "f1c9d382-c26a-463a-a0c2-7871c738697f" (UID: "f1c9d382-c26a-463a-a0c2-7871c738697f"). InnerVolumeSpecName "kube-api-access-qq2nx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:55.354976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.354911 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq2nx\" (UniqueName: \"kubernetes.io/projected/f1c9d382-c26a-463a-a0c2-7871c738697f-kube-api-access-qq2nx\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:55.354976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.354934 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c9d382-c26a-463a-a0c2-7871c738697f-tls-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:55.354976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:55.354945 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c9d382-c26a-463a-a0c2-7871c738697f-dshm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:56.157381 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.157349 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" Jul 11 08:25:56.157889 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.157385 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp" event={"ID":"f1c9d382-c26a-463a-a0c2-7871c738697f","Type":"ContainerDied","Data":"a8588b134eeaed5c821f0ff35a445fead7d4d957f0fa27c526298fc32cde67f5"} Jul 11 08:25:56.157889 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.157447 2572 scope.go:117] "RemoveContainer" containerID="b317c0f6174a87be9df679a9efee5a4568d4084422db11e2ccb1ad604c74e860" Jul 11 08:25:56.179332 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.179304 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:25:56.185066 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.185039 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f178a6/rate-test-model-f178a6-kserve-69c8854c5f-mxxsp"] Jul 11 08:25:56.308157 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.308130 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.327151 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.327130 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863442011456_f8346767-9d72-450d-aabb-968da6179b26/test-curl-7591-139863442011456/0.log" Jul 11 08:25:56.350863 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.350836 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:56.356216 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.356191 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:56.362989 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.362970 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhpd8\" (UniqueName: \"kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8\") pod \"f8346767-9d72-450d-aabb-968da6179b26\" (UID: \"f8346767-9d72-450d-aabb-968da6179b26\") " Jul 11 08:25:56.364953 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.364930 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8" (OuterVolumeSpecName: "kube-api-access-fhpd8") pod "f8346767-9d72-450d-aabb-968da6179b26" (UID: "f8346767-9d72-450d-aabb-968da6179b26"). InnerVolumeSpecName "kube-api-access-fhpd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:56.464287 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.464224 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhpd8\" (UniqueName: \"kubernetes.io/projected/f8346767-9d72-450d-aabb-968da6179b26-kube-api-access-fhpd8\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:25:56.472215 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472188 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:56.472562 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472550 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8346767-9d72-450d-aabb-968da6179b26" containerName="test-curl-7591-139863442011456" Jul 11 08:25:56.472609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472564 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8346767-9d72-450d-aabb-968da6179b26" containerName="test-curl-7591-139863442011456" Jul 11 08:25:56.472609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472575 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c9d382-c26a-463a-a0c2-7871c738697f" containerName="main" Jul 11 08:25:56.472609 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472587 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c9d382-c26a-463a-a0c2-7871c738697f" containerName="main" Jul 11 08:25:56.472727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472660 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8346767-9d72-450d-aabb-968da6179b26" containerName="test-curl-7591-139863442011456" Jul 11 08:25:56.472727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.472669 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c9d382-c26a-463a-a0c2-7871c738697f" containerName="main" Jul 11 08:25:56.475765 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.475751 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.479365 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.479343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:56.564770 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.564739 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9mj\" (UniqueName: \"kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj\") pod \"test-curl-7591-139863442011456\" (UID: \"578febb6-fb41-4d7d-a527-6475ebf9af53\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.665697 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.665650 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9mj\" (UniqueName: \"kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj\") pod \"test-curl-7591-139863442011456\" (UID: \"578febb6-fb41-4d7d-a527-6475ebf9af53\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.674167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.674138 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9mj\" (UniqueName: \"kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj\") pod \"test-curl-7591-139863442011456\" (UID: \"578febb6-fb41-4d7d-a527-6475ebf9af53\") " pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.714036 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.714006 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1c9d382-c26a-463a-a0c2-7871c738697f" path="/var/lib/kubelet/pods/f1c9d382-c26a-463a-a0c2-7871c738697f/volumes" Jul 11 08:25:56.714370 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.714324 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8346767-9d72-450d-aabb-968da6179b26" path="/var/lib/kubelet/pods/f8346767-9d72-450d-aabb-968da6179b26/volumes" Jul 11 08:25:56.786112 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.786093 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:56.935744 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:56.935719 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:56.937487 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:25:56.937456 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod578febb6_fb41_4d7d_a527_6475ebf9af53.slice/crio-1b111b8c0faa52ef7fb1ccd84375ee698909be3d29c07d44b9b7c35a8dbde84b WatchSource:0}: Error finding container 1b111b8c0faa52ef7fb1ccd84375ee698909be3d29c07d44b9b7c35a8dbde84b: Status 404 returned error can't find the container with id 1b111b8c0faa52ef7fb1ccd84375ee698909be3d29c07d44b9b7c35a8dbde84b Jul 11 08:25:57.164254 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:57.164228 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:57.164753 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:57.164269 2572 scope.go:117] "RemoveContainer" containerID="11cbf18152960361deff2ad07debeea6fabc10b4de8bdaf393721e1c45b38530" Jul 11 08:25:57.166771 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:57.166733 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863442011456" event={"ID":"578febb6-fb41-4d7d-a527-6475ebf9af53","Type":"ContainerStarted","Data":"1b111b8c0faa52ef7fb1ccd84375ee698909be3d29c07d44b9b7c35a8dbde84b"} Jul 11 08:25:58.174211 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:58.174169 2572 generic.go:356] "Generic (PLEG): container finished" podID="578febb6-fb41-4d7d-a527-6475ebf9af53" containerID="52036d94b1948ba2470c62bb9b3ad54d53abb243e592107e37387695dacc64fa" exitCode=0 Jul 11 08:25:58.174600 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:58.174248 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7591-139863442011456" event={"ID":"578febb6-fb41-4d7d-a527-6475ebf9af53","Type":"ContainerDied","Data":"52036d94b1948ba2470c62bb9b3ad54d53abb243e592107e37387695dacc64fa"} Jul 11 08:25:59.319325 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.319302 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:25:59.338149 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.338124 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7591-139863442011456_578febb6-fb41-4d7d-a527-6475ebf9af53/test-curl-7591-139863442011456/0.log" Jul 11 08:25:59.361311 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.361276 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:59.367805 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.367773 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7591-139863442011456"] Jul 11 08:25:59.387679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.387654 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb9mj\" (UniqueName: \"kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj\") pod \"578febb6-fb41-4d7d-a527-6475ebf9af53\" (UID: \"578febb6-fb41-4d7d-a527-6475ebf9af53\") " Jul 11 08:25:59.389653 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.389625 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj" (OuterVolumeSpecName: "kube-api-access-rb9mj") pod "578febb6-fb41-4d7d-a527-6475ebf9af53" (UID: "578febb6-fb41-4d7d-a527-6475ebf9af53"). InnerVolumeSpecName "kube-api-access-rb9mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:25:59.488573 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:25:59.488545 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb9mj\" (UniqueName: \"kubernetes.io/projected/578febb6-fb41-4d7d-a527-6475ebf9af53-kube-api-access-rb9mj\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:00.188055 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:00.188025 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b111b8c0faa52ef7fb1ccd84375ee698909be3d29c07d44b9b7c35a8dbde84b" Jul 11 08:26:00.188055 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:00.188028 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:26:00.198152 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:26:00.198113 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7591-139863442011456\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:26:00.713854 ip-10-0-132-10 kubenswrapper[2572]: E0711 08:26:00.713810 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7591-139863442011456\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" pod="openshift-ingress/test-curl-7591-139863442011456" Jul 11 08:26:00.715127 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:00.715099 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" path="/var/lib/kubelet/pods/578febb6-fb41-4d7d-a527-6475ebf9af53/volumes" Jul 11 08:26:14.084818 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:14.084785 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:26:39.083794 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:39.083750 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4qtzf"] Jul 11 08:26:51.329402 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:51.329363 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:26:51.329985 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:51.329595 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" containerID="cri-o://42c211f0d7012ae11872c2c83a197b74477dbb19577acffeaa6d5c5a35c96d12" gracePeriod=30 Jul 11 08:26:51.699084 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:51.698998 2572 patch_prober.go:28] interesting pod/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 11 08:26:51.699084 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:51.699053 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 11 08:26:51.983946 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:51.983872 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7568c9f894-wvbfg_d86947d1-1b27-404c-97e6-043aae55dc77/authorino/0.log" Jul 11 08:26:56.522413 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.522371 2572 generic.go:356] "Generic (PLEG): container finished" podID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerID="42c211f0d7012ae11872c2c83a197b74477dbb19577acffeaa6d5c5a35c96d12" exitCode=0 Jul 11 08:26:56.522792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.522460 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerDied","Data":"42c211f0d7012ae11872c2c83a197b74477dbb19577acffeaa6d5c5a35c96d12"} Jul 11 08:26:56.522792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.522537 2572 scope.go:117] "RemoveContainer" containerID="339ba70f601d9737c00a0774400aed27616034cc5a6246093a0145f1f9f6e928" Jul 11 08:26:56.580309 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.580289 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:26:56.624639 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.624550 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-klwhr_2ffd435a-6d4e-4e17-b12e-2e1c217aa80e/manager/0.log" Jul 11 08:26:56.698202 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698170 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698210 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698236 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698281 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698313 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698396 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698425 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69pc9\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698453 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:26:56.698546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698476 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698505 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert\") pod \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\" (UID: \"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5\") " Jul 11 08:26:56.698999 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698813 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:26:56.698999 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698896 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:26:56.698999 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.698979 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.699207 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.699026 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-workload-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.699207 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.699113 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 08:26:56.699406 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.699381 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:26:56.700637 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.700597 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 08:26:56.700761 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.700654 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 11 08:26:56.700761 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.700671 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:26:56.701212 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.701188 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9" (OuterVolumeSpecName: "kube-api-access-69pc9") pod "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" (UID: "65cef1d4-0602-48ea-8eb7-75ba7d6e75d5"). InnerVolumeSpecName "kube-api-access-69pc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 08:26:56.800200 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800168 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-credential-socket\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800200 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800193 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-data\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800208 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69pc9\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-kube-api-access-69pc9\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800220 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istiod-ca-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800232 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-envoy\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800242 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-podinfo\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.800367 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.800252 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5-istio-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\"" Jul 11 08:26:56.986550 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:56.986527 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zjls2_d92e977b-a6f6-4b3d-92b9-6d6bd33aab07/server/0.log" Jul 11 08:26:57.226607 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.226575 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-5czv2_e3d45bfc-4a26-45d5-985a-83d019112f88/copy-manifests/0.log" Jul 11 08:26:57.232485 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.232465 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-5czv2_e3d45bfc-4a26-45d5-985a-83d019112f88/manager/0.log" Jul 11 08:26:57.528907 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.528880 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" Jul 11 08:26:57.529350 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.528915 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2" event={"ID":"65cef1d4-0602-48ea-8eb7-75ba7d6e75d5","Type":"ContainerDied","Data":"8b75f4bbd35960a5b03bfc78c6190dbf6389cc10fa55d1ec85b19397954439a4"} Jul 11 08:26:57.529350 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.528991 2572 scope.go:117] "RemoveContainer" containerID="42c211f0d7012ae11872c2c83a197b74477dbb19577acffeaa6d5c5a35c96d12" Jul 11 08:26:57.547182 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.547156 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:26:57.552218 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.552194 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-12fff0-openshift-default-656d955f95-8lbh2"] Jul 11 08:26:57.943077 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:57.942996 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29729280-7kfjn_3a2df565-b9e5-4bfb-bf43-14b33829abae/cleanup/0.log" Jul 11 08:26:58.056252 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.056223 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29729295-krkww_9a8b607b-ca76-47cc-a29c-b57e5965dd63/cleanup/0.log" Jul 11 08:26:58.174453 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.174428 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-8bc07405-xf7bh_ac012bab-c750-4e5c-89c6-a923fd124973/revoke-keys/0.log" Jul 11 08:26:58.294645 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.294620 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b0adf7bb-vlqz9_5f21581e-d570-48b0-9730-135657db3b7b/revoke-keys/0.log" Jul 11 08:26:58.623596 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.623522 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-c12c06b3-kb8xq_22af16a3-15fd-43d9-8d75-a23d2636dc15/revoke-keys/0.log" Jul 11 08:26:58.711486 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.711446 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" path="/var/lib/kubelet/pods/65cef1d4-0602-48ea-8eb7-75ba7d6e75d5/volumes" Jul 11 08:26:58.725844 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.725804 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-8da7f7-cr8wm_a4961c3d-7f9b-44e9-bcd0-eaf6b7b7e076/revoke-keys/0.log" Jul 11 08:26:58.942876 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:58.942790 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-7f7e8a77-6bbc4_85e1ae30-cd33-4f29-af40-d58239a20975/revoke-keys/0.log" Jul 11 08:26:59.048064 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:59.048039 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-683f5ca7-vfs4m_666db48b-931c-44a9-80a4-b6f05b96ecbc/revoke-keys/0.log" Jul 11 08:26:59.155199 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:26:59.155174 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-df2809d2-c4fjp_65f62545-f739-4a3f-bd46-730edb97f453/revoke-keys/0.log" Jul 11 08:27:00.227300 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.227272 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/util/0.log" Jul 11 08:27:00.233050 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.233032 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/pull/0.log" Jul 11 08:27:00.238481 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.238459 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/extract/0.log" Jul 11 08:27:00.347862 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.347836 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/util/0.log" Jul 11 08:27:00.354734 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.354692 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/pull/0.log" Jul 11 08:27:00.360023 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.360004 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/extract/0.log" Jul 11 08:27:00.471792 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.471769 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/pull/0.log" Jul 11 08:27:00.477632 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.477582 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/extract/0.log" Jul 11 08:27:00.482828 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.482811 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/util/0.log" Jul 11 08:27:00.599064 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.599036 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/util/0.log" Jul 11 08:27:00.604553 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.604534 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/pull/0.log" Jul 11 08:27:00.609867 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.609851 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/extract/0.log" Jul 11 08:27:00.721374 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:00.721346 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7568c9f894-wvbfg_d86947d1-1b27-404c-97e6-043aae55dc77/authorino/0.log" Jul 11 08:27:01.289903 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:01.289873 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mbbrv_55425168-df33-4415-a2ef-53234dbc2b97/manager/0.log" Jul 11 08:27:01.402381 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:01.402351 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4qtzf_5cdac355-fa73-4646-82be-bffaf49a8633/limitador/0.log" Jul 11 08:27:01.848493 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:01.848464 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf426bdr_890ddd9d-34a6-477a-95a2-4cf5b7284464/istio-proxy/0.log" Jul 11 08:27:02.669758 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:02.669728 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddfc4b4dd-5n58n_a8597305-d702-404e-a359-55030ad011fa/router/0.log" Jul 11 08:27:07.167544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.167511 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dchrv/must-gather-5qpfp"] Jul 11 08:27:07.168106 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168087 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.168195 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168118 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.168195 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168131 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" containerName="test-curl-7591-139863442011456" Jul 11 08:27:07.168195 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168140 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" containerName="test-curl-7591-139863442011456" Jul 11 08:27:07.168348 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168333 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="578febb6-fb41-4d7d-a527-6475ebf9af53" containerName="test-curl-7591-139863442011456" Jul 11 08:27:07.168405 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168349 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.168405 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168362 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.168508 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168480 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.168508 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.168491 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cef1d4-0602-48ea-8eb7-75ba7d6e75d5" containerName="istio-proxy" Jul 11 08:27:07.173147 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.173126 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.175820 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.175792 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dchrv\"/\"kube-root-ca.crt\"" Jul 11 08:27:07.176024 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.176003 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dchrv\"/\"default-dockercfg-xfdth\"" Jul 11 08:27:07.176354 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.176335 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dchrv\"/\"openshift-service-ca.crt\"" Jul 11 08:27:07.178075 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.178053 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/must-gather-5qpfp"] Jul 11 08:27:07.291566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.291539 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5139e1f4-e9e6-499b-842f-4485f1ece0e2-must-gather-output\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.291691 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.291589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f4kt\" (UniqueName: \"kubernetes.io/projected/5139e1f4-e9e6-499b-842f-4485f1ece0e2-kube-api-access-2f4kt\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.392679 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.392654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5139e1f4-e9e6-499b-842f-4485f1ece0e2-must-gather-output\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.392840 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.392722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f4kt\" (UniqueName: \"kubernetes.io/projected/5139e1f4-e9e6-499b-842f-4485f1ece0e2-kube-api-access-2f4kt\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.392979 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.392962 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5139e1f4-e9e6-499b-842f-4485f1ece0e2-must-gather-output\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.401071 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.401050 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f4kt\" (UniqueName: \"kubernetes.io/projected/5139e1f4-e9e6-499b-842f-4485f1ece0e2-kube-api-access-2f4kt\") pod \"must-gather-5qpfp\" (UID: \"5139e1f4-e9e6-499b-842f-4485f1ece0e2\") " pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.483920 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.483855 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/must-gather-5qpfp" Jul 11 08:27:07.619922 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:07.619896 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/must-gather-5qpfp"] Jul 11 08:27:07.622093 ip-10-0-132-10 kubenswrapper[2572]: W0711 08:27:07.622062 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5139e1f4_e9e6_499b_842f_4485f1ece0e2.slice/crio-efb96bbe1b116151e0415cb96ab4e23c6e1564f2d2f44cd0cb1796fa0c62380c WatchSource:0}: Error finding container efb96bbe1b116151e0415cb96ab4e23c6e1564f2d2f44cd0cb1796fa0c62380c: Status 404 returned error can't find the container with id efb96bbe1b116151e0415cb96ab4e23c6e1564f2d2f44cd0cb1796fa0c62380c Jul 11 08:27:08.603310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:08.603264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/must-gather-5qpfp" event={"ID":"5139e1f4-e9e6-499b-842f-4485f1ece0e2","Type":"ContainerStarted","Data":"c65546e268f03c396eb0945df0453546568b9d98b3c94ddd64b7192ef5e41c0e"} Jul 11 08:27:08.603742 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:08.603328 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/must-gather-5qpfp" event={"ID":"5139e1f4-e9e6-499b-842f-4485f1ece0e2","Type":"ContainerStarted","Data":"fafd01fce429e3fd52afa222cf38f5e7b2411c0e347147f659c90c58f65d25ed"} Jul 11 08:27:08.603742 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:08.603349 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/must-gather-5qpfp" event={"ID":"5139e1f4-e9e6-499b-842f-4485f1ece0e2","Type":"ContainerStarted","Data":"efb96bbe1b116151e0415cb96ab4e23c6e1564f2d2f44cd0cb1796fa0c62380c"} Jul 11 08:27:08.620492 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:08.620441 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dchrv/must-gather-5qpfp" podStartSLOduration=0.853051931 podStartE2EDuration="1.620424934s" totalImagesPullingTime="767.373003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 08:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:27:08.617516001 +0000 UTC m=+2918.463504930" watchObservedRunningTime="2026-07-11 08:27:08.620424934 +0000 UTC m=+2918.466413827" Jul 11 08:27:09.843811 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:09.843777 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lhxbc_21b00770-91a4-47b6-ae61-6eacb490ae55/global-pull-secret-syncer/0.log" Jul 11 08:27:10.044140 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:10.044101 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x8plb_27e480f2-4a52-462f-be78-567706460221/konnectivity-agent/0.log" Jul 11 08:27:10.065133 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:10.065103 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-10.ec2.internal_749c499dc7c40abc02010a066fc8c1ea/haproxy/0.log" Jul 11 08:27:15.025879 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.025851 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/extract/0.log" Jul 11 08:27:15.049932 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.049839 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/util/0.log" Jul 11 08:27:15.070107 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.070056 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75944lkk_7f920617-30e7-41f8-9795-567a04711e31/pull/0.log" Jul 11 08:27:15.114233 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.114159 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/extract/0.log" Jul 11 08:27:15.134077 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.133992 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/util/0.log" Jul 11 08:27:15.156612 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.156577 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082dp4_845005a9-54c4-43ef-8ce3-9d622ea894b7/pull/0.log" Jul 11 08:27:15.181259 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.181228 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/extract/0.log" Jul 11 08:27:15.201852 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.201827 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/util/0.log" Jul 11 08:27:15.220727 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.220664 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735wpfm_7de989c1-0185-4c33-bd31-2279c6002bcf/pull/0.log" Jul 11 08:27:15.249799 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.249765 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/extract/0.log" Jul 11 08:27:15.270566 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.270541 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/util/0.log" Jul 11 08:27:15.289858 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.289771 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17zhpz_4f52ce27-bcf2-48fd-9ed6-96b47cf77a06/pull/0.log" Jul 11 08:27:15.661813 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.660838 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7568c9f894-wvbfg_d86947d1-1b27-404c-97e6-043aae55dc77/authorino/0.log" Jul 11 08:27:15.970645 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.970363 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mbbrv_55425168-df33-4415-a2ef-53234dbc2b97/manager/0.log" Jul 11 08:27:15.990728 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:15.990671 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4qtzf_5cdac355-fa73-4646-82be-bffaf49a8633/limitador/0.log" Jul 11 08:27:17.638436 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.638407 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/alertmanager/0.log" Jul 11 08:27:17.659506 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.659464 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/config-reloader/0.log" Jul 11 08:27:17.680668 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.680636 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/kube-rbac-proxy-web/0.log" Jul 11 08:27:17.705337 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.705315 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/kube-rbac-proxy/0.log" Jul 11 08:27:17.726552 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.726526 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/kube-rbac-proxy-metric/0.log" Jul 11 08:27:17.750314 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.750285 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/prom-label-proxy/0.log" Jul 11 08:27:17.782451 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.782418 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb198836-f45a-4cb0-87c2-f4b07c62287f/init-config-reloader/0.log" Jul 11 08:27:17.855976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.855816 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mc4dw_9f1206a3-dacc-4ead-a8ad-9db92a016941/kube-state-metrics/0.log" Jul 11 08:27:17.872506 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.872467 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mc4dw_9f1206a3-dacc-4ead-a8ad-9db92a016941/kube-rbac-proxy-main/0.log" Jul 11 08:27:17.893062 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.892988 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mc4dw_9f1206a3-dacc-4ead-a8ad-9db92a016941/kube-rbac-proxy-self/0.log" Jul 11 08:27:17.916425 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:17.916399 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-576994c976-gwgjv_e5e5a4a9-f884-4c0e-a0d8-62bbbb383b05/metrics-server/0.log" Jul 11 08:27:18.104492 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.104454 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zllhq_25393902-e94b-4db6-9f88-0ccf0a2fb728/node-exporter/0.log" Jul 11 08:27:18.122716 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.122673 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zllhq_25393902-e94b-4db6-9f88-0ccf0a2fb728/kube-rbac-proxy/0.log" Jul 11 08:27:18.146546 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.146467 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zllhq_25393902-e94b-4db6-9f88-0ccf0a2fb728/init-textfile/0.log" Jul 11 08:27:18.167875 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.167842 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lzn6l_ebc58e6b-ce0e-429b-a7e9-1aef707b01fc/kube-rbac-proxy-main/0.log" Jul 11 08:27:18.188570 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.188542 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lzn6l_ebc58e6b-ce0e-429b-a7e9-1aef707b01fc/kube-rbac-proxy-self/0.log" Jul 11 08:27:18.212322 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.212292 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lzn6l_ebc58e6b-ce0e-429b-a7e9-1aef707b01fc/openshift-state-metrics/0.log" Jul 11 08:27:18.534143 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.534107 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfd4dfc88-wz4dg_33cbdf41-5531-448d-a358-636f510b7950/telemeter-client/0.log" Jul 11 08:27:18.556799 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.556768 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfd4dfc88-wz4dg_33cbdf41-5531-448d-a358-636f510b7950/reload/0.log" Jul 11 08:27:18.575118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:18.575068 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfd4dfc88-wz4dg_33cbdf41-5531-448d-a358-636f510b7950/kube-rbac-proxy/0.log" Jul 11 08:27:19.399715 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.399662 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2"] Jul 11 08:27:19.408540 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.408505 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.412192 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.411645 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2"] Jul 11 08:27:19.533363 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.533330 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-podres\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.533639 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.533378 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-proc\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.533639 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.533437 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-lib-modules\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.533639 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.533456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfj2\" (UniqueName: \"kubernetes.io/projected/93629461-1fd3-439c-977c-0ad6ad3d7097-kube-api-access-dbfj2\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.533639 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.533611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-sys\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.634976 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.634933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-sys\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-podres\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-proc\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635167 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635141 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-lib-modules\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635177 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-sys\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635168 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfj2\" (UniqueName: \"kubernetes.io/projected/93629461-1fd3-439c-977c-0ad6ad3d7097-kube-api-access-dbfj2\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635263 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-podres\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635340 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635311 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-proc\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.635537 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.635401 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93629461-1fd3-439c-977c-0ad6ad3d7097-lib-modules\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.645234 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.645207 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfj2\" (UniqueName: \"kubernetes.io/projected/93629461-1fd3-439c-977c-0ad6ad3d7097-kube-api-access-dbfj2\") pod \"perf-node-gather-daemonset-zhmr2\" (UID: \"93629461-1fd3-439c-977c-0ad6ad3d7097\") " pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.725809 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.725721 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:19.920315 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:19.915126 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2"] Jul 11 08:27:20.325952 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.325911 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/1.log" Jul 11 08:27:20.332188 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.332157 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qqfcl_2329a2f5-b6c0-4b11-bfa4-ea93e65ada00/console-operator/2.log" Jul 11 08:27:20.722821 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.722737 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" event={"ID":"93629461-1fd3-439c-977c-0ad6ad3d7097","Type":"ContainerStarted","Data":"1661b20644dc9c40111b4eba6df7cd95ccaa94da23ba9df44245cf3896992ab6"} Jul 11 08:27:20.722821 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.722799 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" event={"ID":"93629461-1fd3-439c-977c-0ad6ad3d7097","Type":"ContainerStarted","Data":"9bca033ddebfc26ce3d1517de40bafcdf33b9ee33ec2ee7401830782c7786364"} Jul 11 08:27:20.723339 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.722938 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:20.740870 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:20.740827 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" podStartSLOduration=1.740812891 podStartE2EDuration="1.740812891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 08:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 08:27:20.738981896 +0000 UTC m=+2930.584970792" watchObservedRunningTime="2026-07-11 08:27:20.740812891 +0000 UTC m=+2930.586801783" Jul 11 08:27:22.260491 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:22.260455 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qzl8f_27009180-8547-4131-8824-d912c8fe24a3/dns/0.log" Jul 11 08:27:22.278075 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:22.278048 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qzl8f_27009180-8547-4131-8824-d912c8fe24a3/kube-rbac-proxy/0.log" Jul 11 08:27:22.339130 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:22.339101 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h7j88_70fd8587-ba55-4187-a314-4fa49e388105/dns-node-resolver/0.log" Jul 11 08:27:22.871903 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:22.871871 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-856c5d9b5c-hgdfv_f96658f9-74c3-42a4-aa52-6667cfe59c26/registry/0.log" Jul 11 08:27:22.938902 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:22.938874 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nk6bk_ded7f277-d188-4a8b-b232-80d7fabde531/node-ca/0.log" Jul 11 08:27:23.762986 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:23.762956 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf426bdr_890ddd9d-34a6-477a-95a2-4cf5b7284464/istio-proxy/0.log" Jul 11 08:27:24.176369 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:24.176288 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddfc4b4dd-5n58n_a8597305-d702-404e-a359-55030ad011fa/router/0.log" Jul 11 08:27:24.846739 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:24.846689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6wnzk_480e6828-e7a8-4ff9-ab8a-8165bdf0a8a8/serve-healthcheck-canary/0.log" Jul 11 08:27:25.338385 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:25.338356 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-r9jgt_358c222b-66c5-423c-9cd1-699f17e9e22c/insights-operator/0.log" Jul 11 08:27:25.338587 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:25.338539 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-r9jgt_358c222b-66c5-423c-9cd1-699f17e9e22c/insights-operator/1.log" Jul 11 08:27:25.356536 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:25.356512 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glsf8_84ea31ae-0ba6-46f5-a661-7bb96290d8ee/kube-rbac-proxy/0.log" Jul 11 08:27:25.374482 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:25.374454 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glsf8_84ea31ae-0ba6-46f5-a661-7bb96290d8ee/exporter/0.log" Jul 11 08:27:25.392994 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:25.392971 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glsf8_84ea31ae-0ba6-46f5-a661-7bb96290d8ee/extractor/0.log" Jul 11 08:27:26.745226 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:26.745194 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dchrv/perf-node-gather-daemonset-zhmr2" Jul 11 08:27:27.572637 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:27.572606 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-klwhr_2ffd435a-6d4e-4e17-b12e-2e1c217aa80e/manager/0.log" Jul 11 08:27:27.888327 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:27.888244 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zjls2_d92e977b-a6f6-4b3d-92b9-6d6bd33aab07/server/0.log" Jul 11 08:27:28.020810 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:28.020775 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-5czv2_e3d45bfc-4a26-45d5-985a-83d019112f88/manager/0.log" Jul 11 08:27:28.040514 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:28.040481 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8b67ddd-5czv2_e3d45bfc-4a26-45d5-985a-83d019112f88/copy-manifests/0.log" Jul 11 08:27:29.522647 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:29.522595 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-697d57cd68-kr86b_a3b09c8e-52c2-4a06-bcb4-357f666dd2e1/manager/0.log" Jul 11 08:27:29.523118 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:29.522764 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf426bdr_890ddd9d-34a6-477a-95a2-4cf5b7284464/istio-proxy/0.log" Jul 11 08:27:29.544326 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:29.544300 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-k4kzg_c9889286-9663-4f42-8a69-af1213b94754/openshift-lws-operator/0.log" Jul 11 08:27:29.884544 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:29.884516 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddfc4b4dd-5n58n_a8597305-d702-404e-a359-55030ad011fa/router/0.log" Jul 11 08:27:31.089885 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:31.089854 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf426bdr_890ddd9d-34a6-477a-95a2-4cf5b7284464/istio-proxy/0.log" Jul 11 08:27:31.461585 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:31.461509 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddfc4b4dd-5n58n_a8597305-d702-404e-a359-55030ad011fa/router/0.log" Jul 11 08:27:34.674694 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:34.674657 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-k4dkb_117717ba-3261-43a0-a194-5f464ce44c48/kube-storage-version-migrator-operator/1.log" Jul 11 08:27:34.675802 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:34.675783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-k4dkb_117717ba-3261-43a0-a194-5f464ce44c48/kube-storage-version-migrator-operator/0.log" Jul 11 08:27:35.561788 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.561763 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/kube-multus-additional-cni-plugins/0.log" Jul 11 08:27:35.580127 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.580099 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/egress-router-binary-copy/0.log" Jul 11 08:27:35.598231 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.598207 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/cni-plugins/0.log" Jul 11 08:27:35.615650 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.615631 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/bond-cni-plugin/0.log" Jul 11 08:27:35.634004 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.633984 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/routeoverride-cni/0.log" Jul 11 08:27:35.652094 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.652074 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/whereabouts-cni-bincopy/0.log" Jul 11 08:27:35.670310 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:35.670288 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jv2k_387f703e-cf0c-4e64-a468-cdb16936e32f/whereabouts-cni/0.log" Jul 11 08:27:36.025316 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:36.025288 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pjrms_2d67ce78-ea2a-47a4-b1f9-58c5d6e786ce/kube-multus/0.log" Jul 11 08:27:36.158265 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:36.158240 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mcqm7_feffa75b-07b6-4079-9ed3-7f56aa5897cc/network-metrics-daemon/0.log" Jul 11 08:27:36.173550 ip-10-0-132-10 kubenswrapper[2572]: I0711 08:27:36.173523 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mcqm7_feffa75b-07b6-4079-9ed3-7f56aa5897cc/kube-rbac-proxy/0.log"