Jul 28 20:45:05.484458 ip-10-0-140-162 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 20:45:05.883214 ip-10-0-140-162 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:45:05.883214 ip-10-0-140-162 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 20:45:05.883214 ip-10-0-140-162 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:45:05.883214 ip-10-0-140-162 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 20:45:05.883214 ip-10-0-140-162 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:45:05.883897 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.883268    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 20:45:05.888617 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888599    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.888617 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888616    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888619    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888623    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888626    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888629    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888637    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888640    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888642    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888645    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888647    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888650    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888652    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888655    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888658    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888661    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888664    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888666    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888669    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888672    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888675    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.888684 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888677    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888680    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888683    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888685    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888688    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888691    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888694    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888697    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888699    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888702    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888704    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888707    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888709    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888712    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888715    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888717    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888720    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888722    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888725    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888727    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.889174 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888729    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888732    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888734    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888736    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888739    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888741    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888744    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888746    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888749    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888751    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888754    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888756    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888760    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888764    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888768    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888771    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888774    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888776    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888779    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888782    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.889661 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888796    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888800    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888804    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888807    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888810    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888813    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888816    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888820    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888822    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888825    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888828    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888830    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888833    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888835    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888837    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888841    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888844    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888847    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888851    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888854    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.890164 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888857    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888860    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888862    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888864    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888868    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888871    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888875    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888878    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888880    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888883    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888886    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888888    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888891    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888893    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888895    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888898    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888900    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.888903    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890266    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.890685 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890275    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890279    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890282    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890284    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890302    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890306    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890309    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890311    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890314    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890317    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890319    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890322    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890324    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890327    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890329    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890332    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890336    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890339    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890342    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.891154 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890347    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890350    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890353    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890356    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890358    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890361    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890364    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890367    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890369    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890372    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890374    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890376    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890379    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890381    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890384    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890387    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890389    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890393    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890397    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890399    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.891627 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890402    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890404    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890407    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890409    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890412    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890414    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890416    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890419    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890422    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890425    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890427    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890430    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890434    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890437    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890439    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890441    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890444    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890447    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890449    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890452    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.892117 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890454    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890456    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890459    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890461    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890464    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890466    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890469    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890472    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890475    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890478    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890481    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890483    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890486    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890489    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890491    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890493    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890496    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890498    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890502    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.892618 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890505    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890508    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890511    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890514    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890517    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890520    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890522    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890525    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890527    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890530    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890532    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890534    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890537    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890539    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890541    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890544    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890546    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890548    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890551    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.893192 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890554    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890951    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890965    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890969    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890974    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890979    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890982    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890986    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890990    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890994    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890997    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.890999    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891002    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891005    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891007    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891010    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891012    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891015    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891017    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.893668 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891020    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891023    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891025    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891028    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891030    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891033    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891037    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891040    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891042    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891045    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891047    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891050    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891052    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891055    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891057    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891061    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891064    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891067    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891070    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891072    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.894156 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891075    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891078    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891080    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891082    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891085    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891087    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891090    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891092    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891095    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891099    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891102    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891105    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891108    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891111    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891113    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891116    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891118    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891121    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891123    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.894652 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891125    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891128    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891130    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891145    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891148    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891151    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891153    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891156    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891159    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891163    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891173    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891177    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891181    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891184    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891187    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891189    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891193    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891195    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891198    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.895103 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891201    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891204    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891207    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891209    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891212    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891214    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891217    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891219    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891222    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891224    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891227    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891229    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891231    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891234    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891236    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891239    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891241    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891244    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891246    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891249    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.895614 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891251    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891255    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891258    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891331    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891339    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891346    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891350    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891354    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891357    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891361    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891365    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891368    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891371    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891375    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891378    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891381    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891384    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891386    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891389    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891392    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891395    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891400    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891402    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891405    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 20:45:05.896098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891408    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891411    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891415    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891417    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891421    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891424    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891427    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891429    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891432    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891435    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891438    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891442    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891447    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891450    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891453    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891455    2567 flags.go:64] FLAG: --enable-server="true"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891458    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891463    2567 flags.go:64] FLAG: --event-burst="100"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891465    2567 flags.go:64] FLAG: --event-qps="50"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891468    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891474    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891477    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891480    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891483    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891486    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 20:45:05.896684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891489    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891492    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891494    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891497    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891500    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891502    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891505    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891508    2567 flags.go:64] FLAG: --feature-gates=""
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891511    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891514    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891517    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891519    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891522    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891525    2567 flags.go:64] FLAG: --help="false"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891528    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-140-162.ec2.internal"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891530    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891535    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891537    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891541    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891544    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891547    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891550    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891553    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 20:45:05.897288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891556    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891558    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891561    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891564    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891568    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891571    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891574    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891577    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891579    2567 flags.go:64] FLAG: --lock-file=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891582    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891585    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891588    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891593    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891596    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891599    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891601    2567 flags.go:64] FLAG: --logging-format="text"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891604    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891608    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891610    2567 flags.go:64] FLAG: --manifest-url=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891613    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891617    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891620    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891623    2567 flags.go:64] FLAG: --max-pods="110"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891626    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891629    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 20:45:05.898026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891633    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891636    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891639    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891642    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891645    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891653    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891656    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891659    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891662    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891665    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891670    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891674    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891677    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891680    2567 flags.go:64] FLAG: --port="10250"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891683    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891686    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07e51987b7ad91197"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891689    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891692    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891695    2567 flags.go:64] FLAG: --register-node="true"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891698    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891701    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891704    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891707    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891709    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891713    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 20:45:05.899021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891716    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891719    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891721    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891724    2567 flags.go:64] FLAG: --runonce="false"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891727    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891730    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891733    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891736    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891740    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891743    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891746    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891749    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891752    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891755    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891758    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891760    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891763    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891766    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891770    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891776    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891779    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891781    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891785    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891788    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891790    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 20:45:05.899692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891793    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891796    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891799    2567 flags.go:64] FLAG: --v="2"
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891803    2567 flags.go:64] FLAG: --version="false"
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891807    2567 flags.go:64] FLAG: --vmodule=""
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891811    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.891814    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891928    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891932    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891935    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891938    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891941    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891943    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891946    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891948    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891954    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891956    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891959    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891961    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891964    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891966    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.900321 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891969    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891972    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891974    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891977    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891981    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891983    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891986    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891988    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891991    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891993    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891996    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.891998    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892000    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892002    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892005    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892009    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892013    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892015    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892018    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.900828 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892021    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892023    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892026    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892029    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892031    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892033    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892036    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892039    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892042    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892045    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892047    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892050    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892052    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892055    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892058    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892060    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892063    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892068    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892071    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.901305 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892073    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892076    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892080    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892083    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892085    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892088    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892091    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892094    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892096    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892099    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892101    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892104    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892107    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892109    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892112    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892114    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892117    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892119    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892122    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.901763 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892125    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892130    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892147    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892150    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892152    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892155    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892158    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892160    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892162    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892165    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892168    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892171    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892174    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892176    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892178    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892181    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892183    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892186    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892188    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892190    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.902247 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892193    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892195    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892197    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892200    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892202    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892205    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892207    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.892209    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.892766    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.900277    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.900296    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.901225    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.901234    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901292    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.902731 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901297    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901301    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901305    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901310    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901313    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901316    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901319    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901322    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901324    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901327    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901330    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901332    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901335    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901337    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901340    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901342    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901345    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901348    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901350    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.903077 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901353    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901355    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901357    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901360    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901363    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901365    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901368    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901371    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901374    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901377    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901380    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901382    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901384    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901387    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901390    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901392    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901395    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901398    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901400    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901403    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.903632 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901405    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901408    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901410    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901412    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901415    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901417    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901420    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901423    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901425    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901427    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901430    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901433    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901435    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901437    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901440    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901442    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901445    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901447    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901450    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901452    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.904149 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901454    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901457    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901461    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901465    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901468    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901471    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901474    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901477    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901479    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901482    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901484    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901487    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901489    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901492    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901494    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901497    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901499    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901502    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901504    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.904635 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901507    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901510    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901513    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901515    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901517    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901520    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901522    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901525    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901527    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901531    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901534    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901537    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901539    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901542    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901545    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901548    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901551    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901553    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901556    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.905085 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901558    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.901563    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901682    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901686    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901690    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901692    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901695    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901698    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901700    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901703    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901705    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901708    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901710    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901713    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901716    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:45:05.905561 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901719    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901722    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901724    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901727    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901731    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901734    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901737    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901740    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901743    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901746    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901749    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901751    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901754    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901756    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901759    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901761    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901764    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901766    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901769    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901772    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:45:05.905924 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901774    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901777    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901779    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901782    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901784    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901786    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901789    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901791    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901794    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901796    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901798    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901801    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901803    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901806    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901809    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901811    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901814    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901816    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901820    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:45:05.906437 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901823    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901826    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901829    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901832    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901834    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901837    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901839    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901842    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901844    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901847    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901850    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901852    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901854    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901857    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901859    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901862    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901865    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901869    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901872    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:45:05.906889 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901874    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901877    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901879    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901881    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901884    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901887    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901889    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901892    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901895    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901897    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901899    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901902    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901904    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901907    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901909    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901911    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901914    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901917    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901919    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:45:05.907415 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901922    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901924    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901926    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901929    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901931    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901934    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901936    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901938    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:05.901941    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.901946    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.902049    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.904204    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.905473    2567 server.go:1019] "Starting client certificate rotation"
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.905577    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:45:05.907867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.905615    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:45:05.908725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.908711    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:45:05.909925 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.909909    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:45:05.921949 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.921929    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 28 20:45:05.923847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.923832    2567 log.go:25] "Validated CRI v1 image API"
Jul 28 20:45:05.925058 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.925043    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 20:45:05.927469 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.927451    2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7c23ea98-e03a-4096-99d6-fa4f89e0d3d2:/dev/nvme0n1p3 ae3b5388-3b3c-4b21-a1cf-35748c5559d7:/dev/nvme0n1p4]
Jul 28 20:45:05.927543 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.927468    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 20:45:05.933640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.933531    2567 manager.go:216] Machine: {Timestamp:2026-07-28 20:45:05.932245375 +0000 UTC m=+0.331302030 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100394 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29b51a490619a51ac2af85b0c4d527 SystemUUID:ec29b51a-4906-19a5-1ac2-af85b0c4d527 BootID:20d1ae81-e817-4185-b4ee-ee03b2af4059 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:3c:ec:44:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:3c:ec:44:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:93:8f:ec:19:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 20:45:05.933640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.933631    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 20:45:05.933773 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.933705    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 20:45:05.936105 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.936082    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 20:45:05.936245 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.936107    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-162.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 20:45:05.936863 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.936852    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 20:45:05.936904 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.936865    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 20:45:05.936904 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.936872    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:45:05.938777 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.938766    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:45:05.939570 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.939561    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 20:45:05.941049 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941034    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:45:05.941287 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941277    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:45:05.941405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941394    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 20:45:05.941434 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941429    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 20:45:05.941460 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941441    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 20:45:05.941460 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941456    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 20:45:05.941526 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941464    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 28 20:45:05.941526 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941472    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 20:45:05.941833 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941822    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:45:05.941871 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.941845    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:45:05.943854 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.943839    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 20:45:05.945689 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.945677    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 20:45:05.945734 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.945707    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 20:45:05.947933 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947921    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947939    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947945    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947951    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947960    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947965    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 20:45:05.947974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947972    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 20:45:05.948131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947977    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 20:45:05.948131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947984    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 20:45:05.948131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947989    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 20:45:05.948131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.947996    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 20:45:05.948131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.948003    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 20:45:05.949451 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.949441    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 20:45:05.949483 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.949452    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 20:45:05.952519 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.952508    2567 server.go:1268] "Started kubelet"
Jul 28 20:45:05.952628 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.952600    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 20:45:05.952725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.952663    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 20:45:05.952788 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.952733    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 20:45:05.953082 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.952939    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 20:45:05.953239 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.953153    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 20:45:05.953239 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.953197    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-162.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 20:45:05.953439 ip-10-0-140-162 systemd[1]: Started Kubernetes Kubelet.
Jul 28 20:45:05.954244 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.954229    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 20:45:05.955114 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.955101    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 20:45:05.956429 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.956391    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 20:45:05.960824 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.960807    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 20:45:05.960909 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.960813    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 20:45:05.960973 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.960908    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:05.961228 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.960285    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-162.ec2.internal.18c68ef9d3eca256  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-162.ec2.internal,UID:ip-10-0-140-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-162.ec2.internal,},FirstTimestamp:2026-07-28 20:45:05.952481878 +0000 UTC m=+0.351538534,LastTimestamp:2026-07-28 20:45:05.952481878 +0000 UTC m=+0.351538534,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-162.ec2.internal,}"
Jul 28 20:45:05.961708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.961603    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 20:45:05.961792 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.961713    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 20:45:05.961792 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.961594    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 20:45:05.961889 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.961863    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 20:45:05.961889 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.961869    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 20:45:05.962525 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.962498    2567 factory.go:55] Registering systemd factory
Jul 28 20:45:05.962895 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.962882    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 28 20:45:05.963285 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.963274    2567 factory.go:153] Registering CRI-O factory
Jul 28 20:45:05.963412 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.963401    2567 factory.go:223] Registration of the crio container factory successfully
Jul 28 20:45:05.963572 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.963542    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 20:45:05.963706 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.963696    2567 factory.go:103] Registering Raw factory
Jul 28 20:45:05.963857 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.963826    2567 manager.go:1196] Started watching for new ooms in manager
Jul 28 20:45:05.964371 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.964348    2567 manager.go:349] Starting recovery of all containers
Jul 28 20:45:05.967830 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.967802    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 20:45:05.967937 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.967920    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 20:45:05.974895 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.974877    2567 manager.go:354] Recovery completed
Jul 28 20:45:05.975169 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.974996    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8qbdn"
Jul 28 20:45:05.978998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.978986    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:05.980869 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.980853    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:05.980942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.980886    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:05.980942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.980902    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:05.981438 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.981424    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:05.981495 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.981426    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 20:45:05.981495 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.981482    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 20:45:05.981559 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.981498    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:45:05.982063 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.982049    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:05.982100 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.982077    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:05.982100 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.982090    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:05.983231 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.983220    2567 policy_none.go:49] "None policy: Start"
Jul 28 20:45:05.983265 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.983236    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 20:45:05.983265 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.983247    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 20:45:05.983456 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:05.983376    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-162.ec2.internal.18c68ef9d59dcb31  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-162.ec2.internal,UID:ip-10-0-140-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-162.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-162.ec2.internal,},FirstTimestamp:2026-07-28 20:45:05.980869425 +0000 UTC m=+0.379926083,LastTimestamp:2026-07-28 20:45:05.980869425 +0000 UTC m=+0.379926083,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-162.ec2.internal,}"
Jul 28 20:45:05.984430 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.984419    2567 policy_none.go:47] "Start"
Jul 28 20:45:05.985434 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:05.985419    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8qbdn"
Jul 28 20:45:06.020664 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.020650    2567 manager.go:337] "Starting Device Plugin manager"
Jul 28 20:45:06.020749 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.020686    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 20:45:06.020749 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.020700    2567 server.go:87] "Starting device plugin registration server"
Jul 28 20:45:06.020931 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.020918    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 20:45:06.021001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.020933    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 20:45:06.021061 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.021015    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 20:45:06.021125 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.021112    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 20:45:06.021201 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.021125    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 20:45:06.021606 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.021585    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 20:45:06.021689 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.021632    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.068077 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.068052    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 20:45:06.071116 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.069448    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 20:45:06.071116 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.069484    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 20:45:06.071116 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.069504    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 20:45:06.071116 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.069537    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 20:45:06.072282 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.072268    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:45:06.121583 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.121563    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:06.122380 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.122366    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:06.122422 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.122395    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:06.122422 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.122405    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:06.122479 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.122427    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.129169 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.129157    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.129214 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.129175    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-162.ec2.internal\": node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.145480 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.145440    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.169807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.169783    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"]
Jul 28 20:45:06.169900 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.169837    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:06.170624 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.170607    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:06.170624 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.170629    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:06.170752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.170639    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:06.171983 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.171972    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:06.172106 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.172093    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.172155 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.172125    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:06.172678 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.172657    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:06.172734 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.172679    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:06.172734 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.172688    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:06.175392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.173954    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:06.175392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.173982    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:06.175392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.174004    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:06.175975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.175957    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.176057 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.175990    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:45:06.176649 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.176634    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:45:06.176748 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.176659    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:45:06.176748 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.176674    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:45:06.198758 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.198736    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-162.ec2.internal\" not found" node="ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.203148 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.203118    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-162.ec2.internal\" not found" node="ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.246335 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.246314    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.347279 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.347250    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.363852 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.363828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.363950 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.363859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.363950 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.363878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66217b0fb27a8d11ff5105a5ffa34535-config\") pod \"kube-apiserver-proxy-ip-10-0-140-162.ec2.internal\" (UID: \"66217b0fb27a8d11ff5105a5ffa34535\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.447747 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.447674    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.465037 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.465117 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465045    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.465117 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66217b0fb27a8d11ff5105a5ffa34535-config\") pod \"kube-apiserver-proxy-ip-10-0-140-162.ec2.internal\" (UID: \"66217b0fb27a8d11ff5105a5ffa34535\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.465233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.465233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465113    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66217b0fb27a8d11ff5105a5ffa34535-config\") pod \"kube-apiserver-proxy-ip-10-0-140-162.ec2.internal\" (UID: \"66217b0fb27a8d11ff5105a5ffa34535\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.465233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.465152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d5eee1a56b131b18e3bafe96ff6f390-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal\" (UID: \"6d5eee1a56b131b18e3bafe96ff6f390\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.501179 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.501160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.505849 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.505831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:06.548621 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.548596    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.649166 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.649119    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.749712 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.749648    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.850229 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.850191    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.904721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.904699    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 20:45:06.905171 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.904858    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 20:45:06.951171 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:06.951132    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:06.989666 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.989631    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 20:40:06 +0000 UTC" deadline="2028-04-13 20:29:03.790244124 +0000 UTC"
Jul 28 20:45:06.989666 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:06.989661    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14999h43m56.800586479s"
Jul 28 20:45:07.043514 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.043479    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:45:07.051362 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:07.051336    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-162.ec2.internal\" not found"
Jul 28 20:45:07.134658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.134638    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:45:07.157015 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:07.156982    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5eee1a56b131b18e3bafe96ff6f390.slice/crio-9f1c8ce13033a153b8ef1347efb685a1c9eee9d595518eaa4ae1253d24183ea6 WatchSource:0}: Error finding container 9f1c8ce13033a153b8ef1347efb685a1c9eee9d595518eaa4ae1253d24183ea6: Status 404 returned error can't find the container with id 9f1c8ce13033a153b8ef1347efb685a1c9eee9d595518eaa4ae1253d24183ea6
Jul 28 20:45:07.161069 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.161054    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:07.161156 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.161102    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:45:07.172378 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.172361    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:45:07.173439 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.173427    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal"
Jul 28 20:45:07.176173 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.176161    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:45:07.183476 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.183462    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:45:07.915952 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:07.915921    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66217b0fb27a8d11ff5105a5ffa34535.slice/crio-a7dcec891c5e6f2f56edbdb6b3011e809df91e824cea6a66235b2a06e9a062e9 WatchSource:0}: Error finding container a7dcec891c5e6f2f56edbdb6b3011e809df91e824cea6a66235b2a06e9a062e9: Status 404 returned error can't find the container with id a7dcec891c5e6f2f56edbdb6b3011e809df91e824cea6a66235b2a06e9a062e9
Jul 28 20:45:07.942417 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.942393    2567 apiserver.go:52] "Watching apiserver"
Jul 28 20:45:07.949881 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.949861    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 20:45:07.952120 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.952088    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2","openshift-cluster-node-tuning-operator/tuned-dtbz8","openshift-multus/multus-jlsh9","openshift-multus/network-metrics-daemon-6nsk8","openshift-ovn-kubernetes/ovnkube-node-76h48","kube-system/global-pull-secret-syncer-sqwjl","openshift-dns/node-resolver-fkqs2","openshift-image-registry/node-ca-9fkq9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal","openshift-multus/multus-additional-cni-plugins-9wgrn","openshift-network-diagnostics/network-check-target-r27kk","openshift-network-operator/iptables-alerter-v5jmg","kube-system/konnectivity-agent-qvnqk"]
Jul 28 20:45:07.953684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.953663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.954756 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.954735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:07.954859 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:07.954807    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:07.955855 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.955821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:07.955961 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:07.955907    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:07.957018 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.956998    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.958021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.957996    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 20:45:07.958132 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.958112    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mzh6r\""
Jul 28 20:45:07.958273 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.958248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:07.958442 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:07.958330    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:07.958707 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.958693    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 20:45:07.959032 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.959010    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.959123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.959065    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 20:45:07.962165 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.961857    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.962165 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.962033    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 20:45:07.962301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.962246    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gn8pp\""
Jul 28 20:45:07.962349 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.962301    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 20:45:07.963318 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.963297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.966473 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.966448    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.966711 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.966677    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:07.966912 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.966894    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.967260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967241    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 20:45:07.967403 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967387    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 20:45:07.967492 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967478    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cwsxq\""
Jul 28 20:45:07.967601 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967579    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 20:45:07.967712 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967630    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 20:45:07.967903 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.967889    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:07.968676 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.968646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:07.968847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.968699    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.969326 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.969305    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gksv2\""
Jul 28 20:45:07.969552 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.969531    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.969906 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.969891    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.970092 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.970079    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4kx8w\""
Jul 28 20:45:07.970299 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.970276    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:07.970418 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.970401    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.970712 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.970672    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 20:45:07.971607 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.971591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:07.971876 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.971595    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.972195 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.972175    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.972760 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.972739    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n9vpq\""
Jul 28 20:45:07.972861 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.972825    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.972963 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.972943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:07.973043 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.972979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-node-log\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973043 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c60473-776e-49c5-970f-91b2e51f8f4b-tmp-dir\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-multus\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973065    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cmx86\""
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-hostroot\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973109    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-ovn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-log-socket\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovn-node-metrics-cert\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-conf-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-netns\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-kubelet-config\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-kubelet\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-var-lib-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973609    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7m9zj\""
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973679    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-netns\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1c60473-776e-49c5-970f-91b2e51f8f4b-hosts-file\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973727    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkwxb\" (UniqueName: \"kubernetes.io/projected/c1c60473-776e-49c5-970f-91b2e51f8f4b-kube-api-access-rkwxb\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:07.973764 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-system-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-multus-certs\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-env-overrides\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62l24\" (UniqueName: \"kubernetes.io/projected/8b13c21f-de91-49f7-897f-d167ffa36e76-kube-api-access-62l24\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.973982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-socket-dir-parent\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-os-release\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-kubelet\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974101    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-bin\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvjtn\" (UniqueName: \"kubernetes.io/projected/7319dd78-0b49-4bde-b1b1-3469bbf3716b-kube-api-access-lvjtn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974151    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-os-release\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-daemon-config\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974268    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-config\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974295    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 20:45:07.974423 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-script-lib\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-cnibin\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974378    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974402    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-etc-kubernetes\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974462    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cni-binary-copy\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-etc-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974508    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-systemd-units\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-systemd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974537    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-k8s-cni-cncf-io\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974588    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-bin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6xx\" (UniqueName: \"kubernetes.io/projected/0721c49d-97a9-40c5-8742-a16ade512302-kube-api-access-fp6xx\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj424\" (UniqueName: \"kubernetes.io/projected/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-kube-api-access-xj424\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:07.975260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974705    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-netd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-system-cni-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-dbus\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974795    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cnibin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.974810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-slash\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.975123    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.975345    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7zrfv\""
Jul 28 20:45:07.975878 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.975374    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 20:45:07.976667 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:07.976647    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:45:08.010930 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.010903    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c7dxj"
Jul 28 20:45:08.019270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.019251    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c7dxj"
Jul 28 20:45:08.062493 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.062472    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 20:45:08.074231 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.074185    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal" event={"ID":"66217b0fb27a8d11ff5105a5ffa34535","Type":"ContainerStarted","Data":"a7dcec891c5e6f2f56edbdb6b3011e809df91e824cea6a66235b2a06e9a062e9"}
Jul 28 20:45:08.074944 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.074924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-script-lib\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.074959    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-cnibin\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.075052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.074989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.075052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-tmp\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.075052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sm26\" (UniqueName: \"kubernetes.io/projected/f95a9632-7f84-4946-a76f-a5add2875618-kube-api-access-7sm26\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-cnibin\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9485831-57b2-4234-9516-63fdc259e799-host\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-etc-kubernetes\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cni-binary-copy\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-etc-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075245    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-systemd\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-sys-fs\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9485831-57b2-4234-9516-63fdc259e799-serviceca\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.075289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075308    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-systemd-units\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-systemd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df5d6258-ff0f-42f4-833f-1fb20561e8d3-iptables-alerter-script\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-k8s-cni-cncf-io\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-bin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075416    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6xx\" (UniqueName: \"kubernetes.io/projected/0721c49d-97a9-40c5-8742-a16ade512302-kube-api-access-fp6xx\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/232a6fb4-b927-4db1-8957-56f14e6256c0-agent-certs\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj424\" (UniqueName: \"kubernetes.io/projected/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-kube-api-access-xj424\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-netd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-system-cni-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-dbus\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cnibin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075568    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-script-lib\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.075811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-slash\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-node-log\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c60473-776e-49c5-970f-91b2e51f8f4b-tmp-dir\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-modprobe-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075648    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-etc-kubernetes\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-multus\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-hostroot\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075677    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-ovn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-log-socket\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovn-node-metrics-cert\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysconfig\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-kubernetes\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-lib-modules\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-conf-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-netns\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-run\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.076808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-dbus\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-sys\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-var-lib-kubelet\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/232a6fb4-b927-4db1-8957-56f14e6256c0-konnectivity-ca\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075903    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-k8s-cni-cncf-io\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-kubelet-config\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-kubelet\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23b1762-9707-464c-a2c1-22ae6e884d18-kubelet-config\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.075971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-var-lib-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cnibin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-host\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-cni-binary-copy\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-slash\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-etc-tuned\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-node-log\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-etc-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.077652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076081    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076108    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-multus\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-hostroot\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076179    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-ovn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thqf\" (UniqueName: \"kubernetes.io/projected/a9485831-57b2-4234-9516-63fdc259e799-kube-api-access-9thqf\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-log-socket\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c60473-776e-49c5-970f-91b2e51f8f4b-tmp-dir\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-cni-bin\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-systemd-units\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-systemd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.076404    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076458    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.076487    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:08.57645939 +0000 UTC m=+2.975516072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:08.078542 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.076586    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.076880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-run-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-netd\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-netns\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.077109    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:08.577086795 +0000 UTC m=+2.976143455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-kubelet\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077133    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-conf-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077135    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-system-cni-dir\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077168    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-var-lib-openvswitch\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-netns\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1c60473-776e-49c5-970f-91b2e51f8f4b-hosts-file\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077177    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-netns\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1c60473-776e-49c5-970f-91b2e51f8f4b-hosts-file\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkwxb\" (UniqueName: \"kubernetes.io/projected/c1c60473-776e-49c5-970f-91b2e51f8f4b-kube-api-access-rkwxb\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pktm9\" (UniqueName: \"kubernetes.io/projected/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kube-api-access-pktm9\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.079135 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077472    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-run-ovn-kubernetes\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077476    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal" event={"ID":"6d5eee1a56b131b18e3bafe96ff6f390","Type":"ContainerStarted","Data":"9f1c8ce13033a153b8ef1347efb685a1c9eee9d595518eaa4ae1253d24183ea6"}
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df5d6258-ff0f-42f4-833f-1fb20561e8d3-host-slash\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-system-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-multus-certs\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077720    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-env-overrides\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-system-cni-dir\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-run-multus-certs\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62l24\" (UniqueName: \"kubernetes.io/projected/8b13c21f-de91-49f7-897f-d167ffa36e76-kube-api-access-62l24\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-socket-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-socket-dir-parent\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.077949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-conf\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-registration-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.079939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-socket-dir-parent\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078082    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-device-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-os-release\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-kubelet\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-bin\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-host-var-lib-kubelet\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvjtn\" (UniqueName: \"kubernetes.io/projected/7319dd78-0b49-4bde-b1b1-3469bbf3716b-kube-api-access-lvjtn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7319dd78-0b49-4bde-b1b1-3469bbf3716b-host-cni-bin\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-os-release\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078339    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b13c21f-de91-49f7-897f-d167ffa36e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-env-overrides\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b13c21f-de91-49f7-897f-d167ffa36e76-os-release\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078406    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-os-release\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wv8\" (UniqueName: \"kubernetes.io/projected/df5d6258-ff0f-42f4-833f-1fb20561e8d3-kube-api-access-h5wv8\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-daemon-config\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-config\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078885    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-multus-daemon-config\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.080609 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.078910    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovnkube-config\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.081228 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.079847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7319dd78-0b49-4bde-b1b1-3469bbf3716b-ovn-node-metrics-cert\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.085477 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.085456    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:08.085582 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.085482    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:08.085582 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.085497    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:08.085582 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.085559    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:08.58554281 +0000 UTC m=+2.984599480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:08.088362 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.088337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj424\" (UniqueName: \"kubernetes.io/projected/a5225bff-3ea6-4b3c-8fc7-443f1cde1177-kube-api-access-xj424\") pod \"multus-jlsh9\" (UID: \"a5225bff-3ea6-4b3c-8fc7-443f1cde1177\") " pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.088788 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.088763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkwxb\" (UniqueName: \"kubernetes.io/projected/c1c60473-776e-49c5-970f-91b2e51f8f4b-kube-api-access-rkwxb\") pod \"node-resolver-fkqs2\" (UID: \"c1c60473-776e-49c5-970f-91b2e51f8f4b\") " pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.089682 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.089629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6xx\" (UniqueName: \"kubernetes.io/projected/0721c49d-97a9-40c5-8742-a16ade512302-kube-api-access-fp6xx\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:08.090750 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.090732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvjtn\" (UniqueName: \"kubernetes.io/projected/7319dd78-0b49-4bde-b1b1-3469bbf3716b-kube-api-access-lvjtn\") pod \"ovnkube-node-76h48\" (UID: \"7319dd78-0b49-4bde-b1b1-3469bbf3716b\") " pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.092219 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.092198    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62l24\" (UniqueName: \"kubernetes.io/projected/8b13c21f-de91-49f7-897f-d167ffa36e76-kube-api-access-62l24\") pod \"multus-additional-cni-plugins-9wgrn\" (UID: \"8b13c21f-de91-49f7-897f-d167ffa36e76\") " pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.179315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thqf\" (UniqueName: \"kubernetes.io/projected/a9485831-57b2-4234-9516-63fdc259e799-kube-api-access-9thqf\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.179315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pktm9\" (UniqueName: \"kubernetes.io/projected/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kube-api-access-pktm9\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df5d6258-ff0f-42f4-833f-1fb20561e8d3-host-slash\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-socket-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df5d6258-ff0f-42f4-833f-1fb20561e8d3-host-slash\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179464    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-socket-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179482    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-conf\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-registration-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-device-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wv8\" (UniqueName: \"kubernetes.io/projected/df5d6258-ff0f-42f4-833f-1fb20561e8d3-kube-api-access-h5wv8\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-registration-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-tmp\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sm26\" (UniqueName: \"kubernetes.io/projected/f95a9632-7f84-4946-a76f-a5add2875618-kube-api-access-7sm26\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179623    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-conf\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.179698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-device-dir\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9485831-57b2-4234-9516-63fdc259e799-host\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-systemd\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-sys-fs\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9485831-57b2-4234-9516-63fdc259e799-serviceca\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179887    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df5d6258-ff0f-42f4-833f-1fb20561e8d3-iptables-alerter-script\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179890    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9485831-57b2-4234-9516-63fdc259e799-host\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-systemd\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-sys-fs\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.179919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/232a6fb4-b927-4db1-8957-56f14e6256c0-agent-certs\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-modprobe-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysconfig\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180092    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-kubernetes\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-lib-modules\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-run\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-sys\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysconfig\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.180340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-var-lib-kubelet\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180357    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-modprobe-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-sysctl-d\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9485831-57b2-4234-9516-63fdc259e799-serviceca\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180413    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-etc-kubernetes\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-sys\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-var-lib-kubelet\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/232a6fb4-b927-4db1-8957-56f14e6256c0-konnectivity-ca\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180464    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-run\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180474    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-host\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-etc-tuned\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-host\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180392    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f95a9632-7f84-4946-a76f-a5add2875618-lib-modules\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.180981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/232a6fb4-b927-4db1-8957-56f14e6256c0-konnectivity-ca\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.181207 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.181034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df5d6258-ff0f-42f4-833f-1fb20561e8d3-iptables-alerter-script\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.182257 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.182233    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-tmp\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.183390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.183366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f95a9632-7f84-4946-a76f-a5add2875618-etc-tuned\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.183550 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.183535    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/232a6fb4-b927-4db1-8957-56f14e6256c0-agent-certs\") pod \"konnectivity-agent-qvnqk\" (UID: \"232a6fb4-b927-4db1-8957-56f14e6256c0\") " pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.190966 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.189685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pktm9\" (UniqueName: \"kubernetes.io/projected/0d2c4b3d-d15e-42c0-9e39-ea54159cd298-kube-api-access-pktm9\") pod \"aws-ebs-csi-driver-node-dtvt2\" (UID: \"0d2c4b3d-d15e-42c0-9e39-ea54159cd298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.191074 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.191049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wv8\" (UniqueName: \"kubernetes.io/projected/df5d6258-ff0f-42f4-833f-1fb20561e8d3-kube-api-access-h5wv8\") pod \"iptables-alerter-v5jmg\" (UID: \"df5d6258-ff0f-42f4-833f-1fb20561e8d3\") " pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.191500 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.191476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sm26\" (UniqueName: \"kubernetes.io/projected/f95a9632-7f84-4946-a76f-a5add2875618-kube-api-access-7sm26\") pod \"tuned-dtbz8\" (UID: \"f95a9632-7f84-4946-a76f-a5add2875618\") " pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.192504 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.192484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thqf\" (UniqueName: \"kubernetes.io/projected/a9485831-57b2-4234-9516-63fdc259e799-kube-api-access-9thqf\") pod \"node-ca-9fkq9\" (UID: \"a9485831-57b2-4234-9516-63fdc259e799\") " pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.269051 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.269017    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9wgrn"
Jul 28 20:45:08.277292 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.277266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jlsh9"
Jul 28 20:45:08.287304 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.287283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:08.292166 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.292131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fkqs2"
Jul 28 20:45:08.299788 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.299764    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9fkq9"
Jul 28 20:45:08.307640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.307621    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dtbz8"
Jul 28 20:45:08.315856 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.315835    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2"
Jul 28 20:45:08.323541 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.323523    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v5jmg"
Jul 28 20:45:08.330206 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.330185    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:08.432186 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.432089    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:45:08.514443 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.514412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5225bff_3ea6_4b3c_8fc7_443f1cde1177.slice/crio-019f5d84693b7a250e7edc30a566c2d18a1e5df55a933fbebd01cc17d72dcd96 WatchSource:0}: Error finding container 019f5d84693b7a250e7edc30a566c2d18a1e5df55a933fbebd01cc17d72dcd96: Status 404 returned error can't find the container with id 019f5d84693b7a250e7edc30a566c2d18a1e5df55a933fbebd01cc17d72dcd96
Jul 28 20:45:08.526968 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.526941    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232a6fb4_b927_4db1_8957_56f14e6256c0.slice/crio-bb830fddab8d530d8087230218cd15a25340fd3ba893d62bd230ba58441f1168 WatchSource:0}: Error finding container bb830fddab8d530d8087230218cd15a25340fd3ba893d62bd230ba58441f1168: Status 404 returned error can't find the container with id bb830fddab8d530d8087230218cd15a25340fd3ba893d62bd230ba58441f1168
Jul 28 20:45:08.527525 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.527485    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9485831_57b2_4234_9516_63fdc259e799.slice/crio-05b05ba931d971c0098cdbd1a2e0b51030e42670acd9bc2f4c9056a5e6527eae WatchSource:0}: Error finding container 05b05ba931d971c0098cdbd1a2e0b51030e42670acd9bc2f4c9056a5e6527eae: Status 404 returned error can't find the container with id 05b05ba931d971c0098cdbd1a2e0b51030e42670acd9bc2f4c9056a5e6527eae
Jul 28 20:45:08.531494 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.531472    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95a9632_7f84_4946_a76f_a5add2875618.slice/crio-eef1dbdc1f3428f6f8d1fb7601b1ff997ffe6bfdd1261fce0e890a73ce66bc32 WatchSource:0}: Error finding container eef1dbdc1f3428f6f8d1fb7601b1ff997ffe6bfdd1261fce0e890a73ce66bc32: Status 404 returned error can't find the container with id eef1dbdc1f3428f6f8d1fb7601b1ff997ffe6bfdd1261fce0e890a73ce66bc32
Jul 28 20:45:08.532843 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.532807    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2c4b3d_d15e_42c0_9e39_ea54159cd298.slice/crio-6d33fee67c0b6fcbb7f4e8c7c17e8af7d29bc51f145ccabece4a8d82de77c775 WatchSource:0}: Error finding container 6d33fee67c0b6fcbb7f4e8c7c17e8af7d29bc51f145ccabece4a8d82de77c775: Status 404 returned error can't find the container with id 6d33fee67c0b6fcbb7f4e8c7c17e8af7d29bc51f145ccabece4a8d82de77c775
Jul 28 20:45:08.533770 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.533467    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5d6258_ff0f_42f4_833f_1fb20561e8d3.slice/crio-af420ec4459c60a22907077bc79b8e3c7276d6519d3a238c288481a63664303a WatchSource:0}: Error finding container af420ec4459c60a22907077bc79b8e3c7276d6519d3a238c288481a63664303a: Status 404 returned error can't find the container with id af420ec4459c60a22907077bc79b8e3c7276d6519d3a238c288481a63664303a
Jul 28 20:45:08.535859 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.535825    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c60473_776e_49c5_970f_91b2e51f8f4b.slice/crio-6b1b6d0b641114f0ec82ccb1a22a151803b472bb47cda019d9e4c2b6b8ced64c WatchSource:0}: Error finding container 6b1b6d0b641114f0ec82ccb1a22a151803b472bb47cda019d9e4c2b6b8ced64c: Status 404 returned error can't find the container with id 6b1b6d0b641114f0ec82ccb1a22a151803b472bb47cda019d9e4c2b6b8ced64c
Jul 28 20:45:08.538186 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.537769    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b13c21f_de91_49f7_897f_d167ffa36e76.slice/crio-360b5f9f05c5afd107d08971bfa40b8baed660a0ee7d5536aed85bdd57077d83 WatchSource:0}: Error finding container 360b5f9f05c5afd107d08971bfa40b8baed660a0ee7d5536aed85bdd57077d83: Status 404 returned error can't find the container with id 360b5f9f05c5afd107d08971bfa40b8baed660a0ee7d5536aed85bdd57077d83
Jul 28 20:45:08.538186 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:08.538128    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7319dd78_0b49_4bde_b1b1_3469bbf3716b.slice/crio-df0b21984fb590cdf48579dcdca69d84b2f7648e5e2f854b7e3dff62b762f77f WatchSource:0}: Error finding container df0b21984fb590cdf48579dcdca69d84b2f7648e5e2f854b7e3dff62b762f77f: Status 404 returned error can't find the container with id df0b21984fb590cdf48579dcdca69d84b2f7648e5e2f854b7e3dff62b762f77f
Jul 28 20:45:08.584489 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.584462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:08.584578 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.584496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:08.584623 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.584610    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:08.584623 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.584610    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:08.584687 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.584668    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:09.584648737 +0000 UTC m=+3.983705382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:08.584687 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.584686    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:09.584676327 +0000 UTC m=+3.983732974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:08.685645 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:08.685565    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:08.685828 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.685718    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:08.685828 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.685752    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:08.685828 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.685767    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:08.686007 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:08.685834    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:09.685813387 +0000 UTC m=+4.084870035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:09.021793 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.021511    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:40:08 +0000 UTC" deadline="2028-05-03 12:48:34.296115312 +0000 UTC"
Jul 28 20:45:09.021793 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.021789    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15472h3m25.274333737s"
Jul 28 20:45:09.083001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.082968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerStarted","Data":"360b5f9f05c5afd107d08971bfa40b8baed660a0ee7d5536aed85bdd57077d83"}
Jul 28 20:45:09.084792 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.084756    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fkqs2" event={"ID":"c1c60473-776e-49c5-970f-91b2e51f8f4b","Type":"ContainerStarted","Data":"6b1b6d0b641114f0ec82ccb1a22a151803b472bb47cda019d9e4c2b6b8ced64c"}
Jul 28 20:45:09.089034 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.089006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v5jmg" event={"ID":"df5d6258-ff0f-42f4-833f-1fb20561e8d3","Type":"ContainerStarted","Data":"af420ec4459c60a22907077bc79b8e3c7276d6519d3a238c288481a63664303a"}
Jul 28 20:45:09.090734 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.090706    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9fkq9" event={"ID":"a9485831-57b2-4234-9516-63fdc259e799","Type":"ContainerStarted","Data":"05b05ba931d971c0098cdbd1a2e0b51030e42670acd9bc2f4c9056a5e6527eae"}
Jul 28 20:45:09.094413 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.094389    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2" event={"ID":"0d2c4b3d-d15e-42c0-9e39-ea54159cd298","Type":"ContainerStarted","Data":"6d33fee67c0b6fcbb7f4e8c7c17e8af7d29bc51f145ccabece4a8d82de77c775"}
Jul 28 20:45:09.095715 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.095693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jlsh9" event={"ID":"a5225bff-3ea6-4b3c-8fc7-443f1cde1177","Type":"ContainerStarted","Data":"019f5d84693b7a250e7edc30a566c2d18a1e5df55a933fbebd01cc17d72dcd96"}
Jul 28 20:45:09.100290 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.100260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"df0b21984fb590cdf48579dcdca69d84b2f7648e5e2f854b7e3dff62b762f77f"}
Jul 28 20:45:09.111431 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.111401    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dtbz8" event={"ID":"f95a9632-7f84-4946-a76f-a5add2875618","Type":"ContainerStarted","Data":"eef1dbdc1f3428f6f8d1fb7601b1ff997ffe6bfdd1261fce0e890a73ce66bc32"}
Jul 28 20:45:09.116863 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.116657    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvnqk" event={"ID":"232a6fb4-b927-4db1-8957-56f14e6256c0","Type":"ContainerStarted","Data":"bb830fddab8d530d8087230218cd15a25340fd3ba893d62bd230ba58441f1168"}
Jul 28 20:45:09.122466 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.121688    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d5eee1a56b131b18e3bafe96ff6f390" containerID="63c0d45b7defc00c7cbd36e77c1a23eda9ffc2c93f3ddb96131d0a8ddeaedafb" exitCode=0
Jul 28 20:45:09.122466 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.121724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal" event={"ID":"6d5eee1a56b131b18e3bafe96ff6f390","Type":"ContainerDied","Data":"63c0d45b7defc00c7cbd36e77c1a23eda9ffc2c93f3ddb96131d0a8ddeaedafb"}
Jul 28 20:45:09.594379 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.594290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:09.594597 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.594394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:09.594597 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.594529    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:09.594597 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.594595    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:11.594574988 +0000 UTC m=+5.993631659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:09.594774 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.594759    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:09.594845 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.594826    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:11.594805467 +0000 UTC m=+5.993862133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:09.695003 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:09.694957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:09.695215 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.695139    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:09.695215 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.695160    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:09.695215 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.695193    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:09.695387 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:09.695265    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:11.695242265 +0000 UTC m=+6.094298911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:10.023685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.022626    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:40:08 +0000 UTC" deadline="2028-01-22 10:57:25.420207888 +0000 UTC"
Jul 28 20:45:10.023685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.022678    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13022h12m15.397546144s"
Jul 28 20:45:10.071356 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.071320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:10.071530 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:10.071456    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:10.071897 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.071856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:10.071991 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:10.071961    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:10.072059 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.072024    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:10.072198 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:10.072175    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:10.143374 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.143336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal" event={"ID":"6d5eee1a56b131b18e3bafe96ff6f390","Type":"ContainerStarted","Data":"4f65b7300bfc0c7ea658f9c3bf20521e8bcfd2e540be25fae0f9a6028fc5b996"}
Jul 28 20:45:10.164966 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:10.164895    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-162.ec2.internal" podStartSLOduration=3.164875729 podStartE2EDuration="3.164875729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:10.164625333 +0000 UTC m=+4.563682000" watchObservedRunningTime="2026-07-28 20:45:10.164875729 +0000 UTC m=+4.563932399"
Jul 28 20:45:11.611937 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:11.611885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:11.612487 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:11.611946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:11.612487 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.612092    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:11.612487 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.612157    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:15.612137801 +0000 UTC m=+10.011194452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:11.612683 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.612591    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:11.612683 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.612639    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:15.612623908 +0000 UTC m=+10.011680556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:11.712591 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:11.712552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:11.712816 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.712795    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:11.712944 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.712820    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:11.712944 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.712830    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:11.712944 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:11.712891    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:15.712872924 +0000 UTC m=+10.111929572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:12.070466 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:12.070433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:12.070649 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:12.070560    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:12.070649 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:12.070642    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:12.070783 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:12.070739    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:12.071096 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:12.071076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:12.071224 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:12.071170    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:14.070444 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:14.070406    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:14.070861 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:14.070534    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:14.070861 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:14.070418    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:14.070861 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:14.070621    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:14.070861 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:14.070407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:14.070861 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:14.070669    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:15.646167 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:15.646117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:15.646687 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:15.646202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:15.646687 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.646368    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:15.646687 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.646431    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:23.646412309 +0000 UTC m=+18.045468961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:15.646864 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.646851    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:15.646920 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.646892    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:23.646881132 +0000 UTC m=+18.045937777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:15.747490 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:15.747444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:15.747661 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.747639    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:15.747661 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.747659    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:15.747738 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.747672    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:15.747792 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:15.747740    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:23.747719433 +0000 UTC m=+18.146776080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:16.070887 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:16.070855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:16.071077 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:16.070943    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:16.071315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:16.071298    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:16.071402 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:16.071386    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:16.071472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:16.071432    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:16.071526 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:16.071478    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:18.070579 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:18.070476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:18.071362 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:18.070684    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:18.071362 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:18.070746    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:18.071362 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:18.070853    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:18.071362 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:18.070509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:18.071362 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:18.070938    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:20.070826 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:20.070778    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:20.071272 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:20.070843    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:20.071272 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:20.070862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:20.071272 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:20.070997    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:20.071272 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:20.071058    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:20.071272 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:20.071199    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:22.070198 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:22.070161    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:22.070198 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:22.070204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:22.070701 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:22.070284    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:22.070701 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:22.070164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:22.070701 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:22.070392    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:22.070701 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:22.070477    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:23.716624 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:23.716581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:23.717129 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:23.716642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:23.717129 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.716760    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:23.717129 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.716760    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:23.717129 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.716821    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:39.716800896 +0000 UTC m=+34.115857541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:23.717129 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.716835    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:39.716829008 +0000 UTC m=+34.115885651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:23.817519 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:23.817472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:23.817708 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.817664    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:23.817708 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.817689    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:23.817708 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.817703    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:23.817818 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:23.817767    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:45:39.817752997 +0000 UTC m=+34.216809640 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:24.070209 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:24.070176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:24.070373 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:24.070177    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:24.070373 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:24.070315    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:24.070373 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:24.070307    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:24.070539 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:24.070426    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:24.070539 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:24.070515    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:26.071375 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:26.071343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:26.071847 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:26.071469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:26.071847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:26.071555    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:26.071847 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:26.071660    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:26.071847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:26.071701    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:26.071847 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:26.071768    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:28.070584 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:28.070548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:28.071164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:28.070590    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:28.071164 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:28.070653    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:28.071164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:28.070548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:28.071164 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:28.070776    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:28.071164 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:28.070837    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:29.189011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.188541    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="9172ef09870a6579d3945630dfc4457591511399a18aae7ae41ac8936d110629" exitCode=0
Jul 28 20:45:29.189811 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.189212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"9172ef09870a6579d3945630dfc4457591511399a18aae7ae41ac8936d110629"}
Jul 28 20:45:29.193098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.193044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fkqs2" event={"ID":"c1c60473-776e-49c5-970f-91b2e51f8f4b","Type":"ContainerStarted","Data":"4bd6c907ddff997011d8213dc68ca2aad87672cd45d4ec3e7ff569135a325e18"}
Jul 28 20:45:29.195624 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.195554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9fkq9" event={"ID":"a9485831-57b2-4234-9516-63fdc259e799","Type":"ContainerStarted","Data":"dfa3b85c9b8f8708fd9fc2418f9fb6095ed904e1b24d9359be977ea793080ba3"}
Jul 28 20:45:29.197130 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.197106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2" event={"ID":"0d2c4b3d-d15e-42c0-9e39-ea54159cd298","Type":"ContainerStarted","Data":"f846eff8a4ccdad80d3b76887ba47a54cd3589b3afb17875d6c775f6a85786dd"}
Jul 28 20:45:29.198687 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.198664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jlsh9" event={"ID":"a5225bff-3ea6-4b3c-8fc7-443f1cde1177","Type":"ContainerStarted","Data":"09cf57e9aebe9db5946b8ec832de422e394e84300a975185c802b39bcedfda75"}
Jul 28 20:45:29.205383 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205362    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"1575795a37074a5bb11600c0abd9dbafcfd37bcc7d3e15bff43696c82ae6ee96"}
Jul 28 20:45:29.205561 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205545    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"61663efc9cfca71d5803a989e6e6f5253bc5ff05c0ce1b47460561f5f9c54da7"}
Jul 28 20:45:29.205748 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"7f1aab285abfd143368e8bcf33ef1504903b028668ec7d5e6a56a6443666a5a2"}
Jul 28 20:45:29.205848 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205784    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"ce92ef0ae5e91869cc140704ad1c574eb23500fcba98f5c5636e8dc70d6922d6"}
Jul 28 20:45:29.205848 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"341a1ddf73836442e5a3ba45dc2dd251f10a5e16b2c14ae43e20414833e500d0"}
Jul 28 20:45:29.205929 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.205875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"a68c734808c943e8d613a85315ceb2b84be0a8254316a2fb7c54ec485bc95c2d"}
Jul 28 20:45:29.206882 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.206856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dtbz8" event={"ID":"f95a9632-7f84-4946-a76f-a5add2875618","Type":"ContainerStarted","Data":"a51a72ec0163314c7c1e3f58c021e62ccedfb881d17b2dbb3208a740a7126655"}
Jul 28 20:45:29.208679 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.208651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvnqk" event={"ID":"232a6fb4-b927-4db1-8957-56f14e6256c0","Type":"ContainerStarted","Data":"9d54e87bf74ee8c0b0fb46ca8d448f3886be47be6d809cf1e8940f4fc27e33b0"}
Jul 28 20:45:29.210227 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.210206    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal" event={"ID":"66217b0fb27a8d11ff5105a5ffa34535","Type":"ContainerStarted","Data":"3b4b74ec751f3ed4c9433d598a0289f972a7f555028bbdaddaf74b3425032b56"}
Jul 28 20:45:29.230201 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.230155    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dtbz8" podStartSLOduration=2.451819945 podStartE2EDuration="22.230123889s" totalImagesPullingTime="19.778303944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.22951235 +0000 UTC m=+23.628569017" watchObservedRunningTime="2026-07-28 20:45:29.230123889 +0000 UTC m=+23.629180556"
Jul 28 20:45:29.246317 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.246274    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qvnqk" podStartSLOduration=2.48447623 podStartE2EDuration="22.246259663s" totalImagesPullingTime="19.761783433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.245792884 +0000 UTC m=+23.644849550" watchObservedRunningTime="2026-07-28 20:45:29.246259663 +0000 UTC m=+23.645316330"
Jul 28 20:45:29.261714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.261681    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-162.ec2.internal" podStartSLOduration=21.261667808 podStartE2EDuration="21.261667808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.261491376 +0000 UTC m=+23.660548041" watchObservedRunningTime="2026-07-28 20:45:29.261667808 +0000 UTC m=+23.660724473"
Jul 28 20:45:29.276952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.276919    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fkqs2" podStartSLOduration=2.524789053 podStartE2EDuration="22.276908443s" totalImagesPullingTime="19.75211939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.276447122 +0000 UTC m=+23.675503789" watchObservedRunningTime="2026-07-28 20:45:29.276908443 +0000 UTC m=+23.675965108"
Jul 28 20:45:29.311956 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.311917    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9fkq9" podStartSLOduration=2.549803862 podStartE2EDuration="22.311902754s" totalImagesPullingTime="19.762098892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.290991412 +0000 UTC m=+23.690048076" watchObservedRunningTime="2026-07-28 20:45:29.311902754 +0000 UTC m=+23.710959418"
Jul 28 20:45:29.312288 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.312266    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jlsh9" podStartSLOduration=2.483142619 podStartE2EDuration="22.312260796s" totalImagesPullingTime="19.829118177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.311788527 +0000 UTC m=+23.710845191" watchObservedRunningTime="2026-07-28 20:45:29.312260796 +0000 UTC m=+23.711317461"
Jul 28 20:45:29.910011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:29.909988    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 20:45:30.031308 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.031212    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T20:45:29.910006962Z","UUID":"35a3b5d1-c178-47c8-814c-95e19626c33d","Handler":null,"Name":"","Endpoint":""}
Jul 28 20:45:30.033510 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.033490    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 20:45:30.033636 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.033521    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 20:45:30.070752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.070687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:30.070868 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.070689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:30.070868 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:30.070816    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:30.070962 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:30.070898    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:30.070962 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.070949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:30.071081 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:30.071062    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:30.214417 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.214381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v5jmg" event={"ID":"df5d6258-ff0f-42f4-833f-1fb20561e8d3","Type":"ContainerStarted","Data":"90bce454f8d9c125a8ee41a3c18cd03324c4b852bb3a182a4b74524900caf83a"}
Jul 28 20:45:30.216494 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.216467    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2" event={"ID":"0d2c4b3d-d15e-42c0-9e39-ea54159cd298","Type":"ContainerStarted","Data":"caa8e42995ed912c81aac2fc375d6fb437500c867559526d3f210d070f7dc29a"}
Jul 28 20:45:30.231643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:30.231603    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v5jmg" podStartSLOduration=3.478368589 podStartE2EDuration="23.231588281s" totalImagesPullingTime="19.753219692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:30.230978123 +0000 UTC m=+24.630034789" watchObservedRunningTime="2026-07-28 20:45:30.231588281 +0000 UTC m=+24.630644945"
Jul 28 20:45:31.033080 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:31.033054    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:31.033602 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:31.033574    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:31.222859 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:31.222822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"f7c74430d6101d430994903c700e3e06fa4486f46bd906df5a192674b75264c0"}
Jul 28 20:45:31.223392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:31.223179    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:31.223728 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:31.223704    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qvnqk"
Jul 28 20:45:32.070901 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:32.070726    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:32.071085 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:32.070725    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:32.071085 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:32.071001    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:32.071085 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:32.070726    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:32.071085 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:32.071076    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:32.071383 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:32.071175    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:34.070470 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.070294    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:34.071189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.070299    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:34.071189 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:34.070549    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:34.071189 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:34.070640    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:34.071189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.070299    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:34.071189 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:34.070754    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:34.230598 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.230514    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="753c32b772149a8e6826da76c106c1bef5e4b25310b17233a78fc0f3d43b192d" exitCode=0
Jul 28 20:45:34.230598 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.230589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"753c32b772149a8e6826da76c106c1bef5e4b25310b17233a78fc0f3d43b192d"}
Jul 28 20:45:34.232805 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.232775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2" event={"ID":"0d2c4b3d-d15e-42c0-9e39-ea54159cd298","Type":"ContainerStarted","Data":"88f7dd4e170ba54a0f585de14f77b7bbd3216c5bd5d83af8d55f60c2b9b1085f"}
Jul 28 20:45:34.236345 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.236325    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" event={"ID":"7319dd78-0b49-4bde-b1b1-3469bbf3716b","Type":"ContainerStarted","Data":"1657c2193d31c631b96c7bb5291541a024c8c143ec84d03b2f3aa3560a718885"}
Jul 28 20:45:34.236527 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.236508    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:34.236596 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.236580    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:34.252767 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.252750    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:34.353431 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:34.353380    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-76h48" podStartSLOduration=7.509295193 podStartE2EDuration="27.353364994s" totalImagesPullingTime="19.844069801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:34.295980887 +0000 UTC m=+28.695037552" watchObservedRunningTime="2026-07-28 20:45:34.353364994 +0000 UTC m=+28.752421688"
Jul 28 20:45:35.239015 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:35.238988    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:35.253963 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:35.253938    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:45:35.292209 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:35.292126    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtvt2" podStartSLOduration=3.6707257049999997 podStartE2EDuration="28.292108616s" totalImagesPullingTime="24.621382911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:34.35369794 +0000 UTC m=+28.752754605" watchObservedRunningTime="2026-07-28 20:45:35.292108616 +0000 UTC m=+29.691165281"
Jul 28 20:45:36.070897 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.070863    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:36.071060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.070963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:36.071060 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.070984    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:36.071060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.071044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:36.071060 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.071045    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:36.071234 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.071119    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:36.222584 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.222557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sqwjl"]
Jul 28 20:45:36.227012 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.226984    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6nsk8"]
Jul 28 20:45:36.227637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.227616    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r27kk"]
Jul 28 20:45:36.242752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.242696    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="7105522c11191e15a47c9c8c0806a977e347140597b0c5d61231ddf568cd10e0" exitCode=0
Jul 28 20:45:36.243086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.242811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"7105522c11191e15a47c9c8c0806a977e347140597b0c5d61231ddf568cd10e0"}
Jul 28 20:45:36.243086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.242855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:36.243086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.242935    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:36.243086 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.243033    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:36.243086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:36.243069    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:36.243288 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.243199    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:36.243642 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:36.243616    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:38.070091 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:38.069919    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:38.070410 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:38.069920    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:38.070410 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:38.070199    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:38.070410 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:38.070238    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:38.070410 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:38.069952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:38.070410 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:38.070306    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:38.249442 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:38.249371    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="c0add1f59c7c63552d6f5405c04027ab1a9d4ff893f1cb1679b7b47d74d2526e" exitCode=0
Jul 28 20:45:38.249442 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:38.249434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"c0add1f59c7c63552d6f5405c04027ab1a9d4ff893f1cb1679b7b47d74d2526e"}
Jul 28 20:45:39.746055 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:39.746026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:39.746564 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:39.746074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:39.746564 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.746210    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:39.746564 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.746212    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:39.746564 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.746276    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs podName:0721c49d-97a9-40c5-8742-a16ade512302 nodeName:}" failed. No retries permitted until 2026-07-28 20:46:11.746255793 +0000 UTC m=+66.145312442 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs") pod "network-metrics-daemon-6nsk8" (UID: "0721c49d-97a9-40c5-8742-a16ade512302") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:45:39.746564 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.746293    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret podName:b23b1762-9707-464c-a2c1-22ae6e884d18 nodeName:}" failed. No retries permitted until 2026-07-28 20:46:11.746284406 +0000 UTC m=+66.145341053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret") pod "global-pull-secret-syncer-sqwjl" (UID: "b23b1762-9707-464c-a2c1-22ae6e884d18") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:45:39.846773 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:39.846745    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:39.846954 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.846940    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:45:39.847017 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.846967    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:45:39.847017 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.846981    2567 projected.go:196] Error preparing data for projected volume kube-api-access-xnh4g for pod openshift-network-diagnostics/network-check-target-r27kk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:39.847115 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:39.847033    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g podName:3512fdc5-10a8-4257-ac49-9d20b7b338c7 nodeName:}" failed. No retries permitted until 2026-07-28 20:46:11.847020678 +0000 UTC m=+66.246077325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnh4g" (UniqueName: "kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g") pod "network-check-target-r27kk" (UID: "3512fdc5-10a8-4257-ac49-9d20b7b338c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:45:40.069845 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.069812    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:40.070029 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.069927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:40.070029 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.069952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:40.070165 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:40.069924    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r27kk" podUID="3512fdc5-10a8-4257-ac49-9d20b7b338c7"
Jul 28 20:45:40.070165 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:40.070044    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nsk8" podUID="0721c49d-97a9-40c5-8742-a16ade512302"
Jul 28 20:45:40.070165 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:40.070130    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sqwjl" podUID="b23b1762-9707-464c-a2c1-22ae6e884d18"
Jul 28 20:45:40.905369 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.905335    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-162.ec2.internal" event="NodeReady"
Jul 28 20:45:40.905874 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.905500    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 20:45:40.944437 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.944400    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:45:40.949658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.949633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:40.951871 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.951850    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kskpf\""
Jul 28 20:45:40.952073 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.952053    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 20:45:40.952222 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.952206    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 20:45:40.952354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.952338    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 20:45:40.962771 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.962747    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4cfcq"]
Jul 28 20:45:40.963668 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.963635    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 20:45:40.967270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.967251    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cldn8"]
Jul 28 20:45:40.967383 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.967366    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:40.969322 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.969304    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 20:45:40.969412 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.969306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 20:45:40.969456 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.969412    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph6jd\""
Jul 28 20:45:40.970703 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.970683    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:45:40.970788 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.970776    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:40.972833 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.972816    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 20:45:40.972925 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.972868    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kmbt4\""
Jul 28 20:45:40.972925 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.972876    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 20:45:40.972925 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.972881    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 20:45:40.981392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.981374    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cldn8"]
Jul 28 20:45:40.981910 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:40.981891    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4cfcq"]
Jul 28 20:45:41.054405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054600    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghfm\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.054882 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.054683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155468 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155468 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155433    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155468 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98db2d83-d062-4737-8e53-c31ba8cb72ad-tmp-dir\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c35c4f86-4695-46f1-911e-f4bdb215b4f1-cert\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghfm\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98db2d83-d062-4737-8e53-c31ba8cb72ad-config-volume\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fww6l\" (UniqueName: \"kubernetes.io/projected/98db2d83-d062-4737-8e53-c31ba8cb72ad-kube-api-access-fww6l\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155662    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzxd\" (UniqueName: \"kubernetes.io/projected/c35c4f86-4695-46f1-911e-f4bdb215b4f1-kube-api-access-5wzxd\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.155685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.155684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.156368 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.156207    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:41.156368 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.156232    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:41.156368 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.156310    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:45:41.656288614 +0000 UTC m=+36.055345258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:41.156637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.156619    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.156890 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.156871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.156988 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.156967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.160195 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.160173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.160195 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.160184    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.167089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.167067    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.167793 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.167774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghfm\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.256174 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98db2d83-d062-4737-8e53-c31ba8cb72ad-tmp-dir\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.256174 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c35c4f86-4695-46f1-911e-f4bdb215b4f1-cert\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.256388 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98db2d83-d062-4737-8e53-c31ba8cb72ad-config-volume\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.256388 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.256388 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fww6l\" (UniqueName: \"kubernetes.io/projected/98db2d83-d062-4737-8e53-c31ba8cb72ad-kube-api-access-fww6l\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.256388 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.256336    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:41.256547 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.256422    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:45:41.756399592 +0000 UTC m=+36.155456238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:45:41.256547 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98db2d83-d062-4737-8e53-c31ba8cb72ad-tmp-dir\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.256648 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256546    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzxd\" (UniqueName: \"kubernetes.io/projected/c35c4f86-4695-46f1-911e-f4bdb215b4f1-kube-api-access-5wzxd\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.256856 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.256829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98db2d83-d062-4737-8e53-c31ba8cb72ad-config-volume\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.259117 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.259095    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c35c4f86-4695-46f1-911e-f4bdb215b4f1-cert\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.266836 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.266813    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzxd\" (UniqueName: \"kubernetes.io/projected/c35c4f86-4695-46f1-911e-f4bdb215b4f1-kube-api-access-5wzxd\") pod \"ingress-canary-cldn8\" (UID: \"c35c4f86-4695-46f1-911e-f4bdb215b4f1\") " pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.267497 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.267478    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fww6l\" (UniqueName: \"kubernetes.io/projected/98db2d83-d062-4737-8e53-c31ba8cb72ad-kube-api-access-fww6l\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.286746 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.286727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cldn8"
Jul 28 20:45:41.659744 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.659707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:41.659908 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.659873    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:41.659908 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.659891    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:41.660018 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.659965    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:45:42.659950241 +0000 UTC m=+37.059006903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:41.760875 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:41.760828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:41.761031 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.761000    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:41.761091 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:41.761079    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:45:42.761059077 +0000 UTC m=+37.160115726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:45:42.069952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.069919    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:45:42.069952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.069937    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:45:42.069952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.069919    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:45:42.072807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.072782    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:45:42.072935 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.072840    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5747k\""
Jul 28 20:45:42.073060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.073045    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:45:42.073299 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.073279    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:45:42.073432 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.073304    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2qcwd\""
Jul 28 20:45:42.073432 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.072780    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:45:42.443400 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.443204    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cldn8"]
Jul 28 20:45:42.447797 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:45:42.447770    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35c4f86_4695_46f1_911e_f4bdb215b4f1.slice/crio-3139e7bb21fd3d3cbfe524aac8e69ec402c8d83d34c2cfe149c0cb1cdf57c8cb WatchSource:0}: Error finding container 3139e7bb21fd3d3cbfe524aac8e69ec402c8d83d34c2cfe149c0cb1cdf57c8cb: Status 404 returned error can't find the container with id 3139e7bb21fd3d3cbfe524aac8e69ec402c8d83d34c2cfe149c0cb1cdf57c8cb
Jul 28 20:45:42.669434 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.669314    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:42.669585 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:42.669435    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:42.669585 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:42.669447    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:42.669585 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:42.669495    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:45:44.66948264 +0000 UTC m=+39.068539283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:42.770041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:42.770003    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:42.770232 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:42.770181    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:42.770275 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:42.770242    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:45:44.770226238 +0000 UTC m=+39.169282885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:45:43.265049 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:43.265009    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="916eda37357dc17e00784c7e9db769ab71117afcc919c306c986506c091f9c40" exitCode=0
Jul 28 20:45:43.265504 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:43.265092    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"916eda37357dc17e00784c7e9db769ab71117afcc919c306c986506c091f9c40"}
Jul 28 20:45:43.266654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:43.266626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cldn8" event={"ID":"c35c4f86-4695-46f1-911e-f4bdb215b4f1","Type":"ContainerStarted","Data":"3139e7bb21fd3d3cbfe524aac8e69ec402c8d83d34c2cfe149c0cb1cdf57c8cb"}
Jul 28 20:45:44.273233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:44.273194    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b13c21f-de91-49f7-897f-d167ffa36e76" containerID="b4d4cfc49035356ec85b15c0de6f39247db6ad4bfa51b5254963662d54f1310d" exitCode=0
Jul 28 20:45:44.273743 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:44.273239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerDied","Data":"b4d4cfc49035356ec85b15c0de6f39247db6ad4bfa51b5254963662d54f1310d"}
Jul 28 20:45:44.686148 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:44.686056    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:44.686274 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:44.686193    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:44.686274 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:44.686207    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:44.686274 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:44.686262    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:45:48.686245693 +0000 UTC m=+43.085302338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:44.786452 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:44.786422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:44.786592 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:44.786573    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:44.786646 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:44.786637    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:45:48.786621323 +0000 UTC m=+43.185677966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:45:45.277385 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:45.277168    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cldn8" event={"ID":"c35c4f86-4695-46f1-911e-f4bdb215b4f1","Type":"ContainerStarted","Data":"c2eaf1f602380bfd6a5188c728878687c311bf575f9c55bcd8dc4f5924646678"}
Jul 28 20:45:45.280556 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:45.280458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" event={"ID":"8b13c21f-de91-49f7-897f-d167ffa36e76","Type":"ContainerStarted","Data":"5146f74911b8cecb29813f238df647e6e9736066657b3f6616bc9577ef4c0b44"}
Jul 28 20:45:45.296845 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:45.296797    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cldn8" podStartSLOduration=2.853655259 podStartE2EDuration="5.296781411s" totalImagesPullingTime="2.443126152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:45.295522245 +0000 UTC m=+39.694578910" watchObservedRunningTime="2026-07-28 20:45:45.296781411 +0000 UTC m=+39.695838075"
Jul 28 20:45:45.320871 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:45.320809    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9wgrn" podStartSLOduration=8.322373788 podStartE2EDuration="39.320788959s" totalImagesPullingTime="30.998415171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:45.319793008 +0000 UTC m=+39.718849672" watchObservedRunningTime="2026-07-28 20:45:45.320788959 +0000 UTC m=+39.719845625"
Jul 28 20:45:48.714809 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.714772    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5ddf7"]
Jul 28 20:45:48.717058 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.717026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:48.717210 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.717193    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:48.717261 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.717212    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:48.717298 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.717268    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:45:56.717250342 +0000 UTC m=+51.116306984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:48.741941 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.741915    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5ddf7"]
Jul 28 20:45:48.742053 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.742040    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.744356 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.744327    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 20:45:48.744479 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.744390    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nwhvj\""
Jul 28 20:45:48.744975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.744950    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 20:45:48.745071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.744980    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 20:45:48.745071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.744992    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 20:45:48.817952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.817914    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.818120 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.817960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9j5t\" (UniqueName: \"kubernetes.io/projected/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-api-access-j9j5t\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.818120 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.818078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-data-volume\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.818228 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.818120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:48.818228 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.818193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-crio-socket\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.818228 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.818210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.818370 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.818303    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:48.818403 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.818372    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:45:56.818356457 +0000 UTC m=+51.217413103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:45:48.918701 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-data-volume\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.918847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-crio-socket\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.918847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.918847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.918847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918802    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9j5t\" (UniqueName: \"kubernetes.io/projected/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-api-access-j9j5t\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.918899    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.918920    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.918936    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-crio-socket\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.918985    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:49.418964465 +0000 UTC m=+43.818021130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : configmap "kube-rbac-proxy" not found
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:48.919006    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:49.418997709 +0000 UTC m=+43.818054351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:48.919034 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.919025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-data-volume\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:48.930635 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:48.930615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9j5t\" (UniqueName: \"kubernetes.io/projected/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-api-access-j9j5t\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:49.421693 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:49.421656    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:49.421869 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:49.421706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:49.421869 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:49.421816    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:49.421869 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:49.421839    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 20:45:49.421961 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:49.421885    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:50.421867702 +0000 UTC m=+44.820924346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:49.421961 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:49.421905    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:50.421892237 +0000 UTC m=+44.820948881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : configmap "kube-rbac-proxy" not found
Jul 28 20:45:50.429539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:50.429503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:50.429925 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:50.429550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:50.429925 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:50.429658    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 20:45:50.429925 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:50.429658    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:50.429925 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:50.429720    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:52.429699371 +0000 UTC m=+46.828756017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : configmap "kube-rbac-proxy" not found
Jul 28 20:45:50.429925 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:50.429779    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:52.429757408 +0000 UTC m=+46.828814051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:50.694370 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:50.694288    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fkqs2_c1c60473-776e-49c5-970f-91b2e51f8f4b/dns-node-resolver/0.log"
Jul 28 20:45:51.693957 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:51.693928    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9fkq9_a9485831-57b2-4234-9516-63fdc259e799/node-ca/0.log"
Jul 28 20:45:52.294467 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:52.294435    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cldn8_c35c4f86-4695-46f1-911e-f4bdb215b4f1/serve-healthcheck-canary/0.log"
Jul 28 20:45:52.446904 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:52.446870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:52.447051 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:52.446916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:52.447051 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:52.447010    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 20:45:52.447051 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:52.447025    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:52.447166 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:52.447076    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:56.447057901 +0000 UTC m=+50.846114545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:52.447166 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:52.447093    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:45:56.447084299 +0000 UTC m=+50.846140942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : configmap "kube-rbac-proxy" not found
Jul 28 20:45:56.478049 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:56.478015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:56.478450 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:56.478060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:45:56.478450 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.478176    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 20:45:56.478450 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.478194    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:56.478450 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.478243    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:46:04.478224989 +0000 UTC m=+58.877281654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : configmap "kube-rbac-proxy" not found
Jul 28 20:45:56.478450 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.478258    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:46:04.478252396 +0000 UTC m=+58.877309038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:45:56.782025 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:56.781986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:45:56.782213 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.782162    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 20:45:56.782213 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.782182    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c6bc9548-nfbrn: secret "image-registry-tls" not found
Jul 28 20:45:56.782290 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.782244    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls podName:82eebc70-3a60-434f-a7bd-e7012bf0157a nodeName:}" failed. No retries permitted until 2026-07-28 20:46:12.782227557 +0000 UTC m=+67.181284219 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls") pod "image-registry-78c6bc9548-nfbrn" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a") : secret "image-registry-tls" not found
Jul 28 20:45:56.882807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:45:56.882775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:45:56.882940 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.882926    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 20:45:56.882993 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:45:56.882984    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls podName:98db2d83-d062-4737-8e53-c31ba8cb72ad nodeName:}" failed. No retries permitted until 2026-07-28 20:46:12.882968438 +0000 UTC m=+67.282025082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls") pod "dns-default-4cfcq" (UID: "98db2d83-d062-4737-8e53-c31ba8cb72ad") : secret "dns-default-metrics-tls" not found
Jul 28 20:46:04.538522 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:04.538462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:04.538522 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:04.538522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:04.539030 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:04.538629    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 20:46:04.539030 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:04.538715    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls podName:514b3bd1-2f12-4d4a-b437-63d4e629fcfd nodeName:}" failed. No retries permitted until 2026-07-28 20:46:20.538696786 +0000 UTC m=+74.937753449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ddf7" (UID: "514b3bd1-2f12-4d4a-b437-63d4e629fcfd") : secret "insights-runtime-extractor-tls" not found
Jul 28 20:46:04.539030 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:04.538975    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:07.259583 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:07.259553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76h48"
Jul 28 20:46:11.793014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.792961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:46:11.793014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.793019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:46:11.795650 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.795627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:46:11.795695 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.795648    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:46:11.805643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.805621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0721c49d-97a9-40c5-8742-a16ade512302-metrics-certs\") pod \"network-metrics-daemon-6nsk8\" (UID: \"0721c49d-97a9-40c5-8742-a16ade512302\") " pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:46:11.805754 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.805675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23b1762-9707-464c-a2c1-22ae6e884d18-original-pull-secret\") pod \"global-pull-secret-syncer-sqwjl\" (UID: \"b23b1762-9707-464c-a2c1-22ae6e884d18\") " pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:46:11.893490 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.893459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:46:11.895952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.895934    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:46:11.906221 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.906203    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:46:11.916728 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:11.916707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnh4g\" (UniqueName: \"kubernetes.io/projected/3512fdc5-10a8-4257-ac49-9d20b7b338c7-kube-api-access-xnh4g\") pod \"network-check-target-r27kk\" (UID: \"3512fdc5-10a8-4257-ac49-9d20b7b338c7\") " pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:46:12.083663 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.083595    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2qcwd\""
Jul 28 20:46:12.089388 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.089362    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5747k\""
Jul 28 20:46:12.091943 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.091930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nsk8"
Jul 28 20:46:12.094199 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.094181    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sqwjl"
Jul 28 20:46:12.098086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.098068    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:46:12.253543 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.253514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sqwjl"]
Jul 28 20:46:12.256856 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:12.256830    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23b1762_9707_464c_a2c1_22ae6e884d18.slice/crio-81b9dcf76a6358d970be01bcf4008caaa0c804887b7e014299c5c03826aaa11e WatchSource:0}: Error finding container 81b9dcf76a6358d970be01bcf4008caaa0c804887b7e014299c5c03826aaa11e: Status 404 returned error can't find the container with id 81b9dcf76a6358d970be01bcf4008caaa0c804887b7e014299c5c03826aaa11e
Jul 28 20:46:12.267682 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.267660    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r27kk"]
Jul 28 20:46:12.270075 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:12.270052    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3512fdc5_10a8_4257_ac49_9d20b7b338c7.slice/crio-a119da18ae80bc7c83d0519dce01de951db0dec3626492f3a5ae9b7792573e1e WatchSource:0}: Error finding container a119da18ae80bc7c83d0519dce01de951db0dec3626492f3a5ae9b7792573e1e: Status 404 returned error can't find the container with id a119da18ae80bc7c83d0519dce01de951db0dec3626492f3a5ae9b7792573e1e
Jul 28 20:46:12.282213 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.282190    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6nsk8"]
Jul 28 20:46:12.284952 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:12.284918    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0721c49d_97a9_40c5_8742_a16ade512302.slice/crio-da594e6adea3e6d675e3cd1231a62f857a618ac26bf38b50fc48edd2491db4ac WatchSource:0}: Error finding container da594e6adea3e6d675e3cd1231a62f857a618ac26bf38b50fc48edd2491db4ac: Status 404 returned error can't find the container with id da594e6adea3e6d675e3cd1231a62f857a618ac26bf38b50fc48edd2491db4ac
Jul 28 20:46:12.342913 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.342856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r27kk" event={"ID":"3512fdc5-10a8-4257-ac49-9d20b7b338c7","Type":"ContainerStarted","Data":"a119da18ae80bc7c83d0519dce01de951db0dec3626492f3a5ae9b7792573e1e"}
Jul 28 20:46:12.344024 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.344000    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sqwjl" event={"ID":"b23b1762-9707-464c-a2c1-22ae6e884d18","Type":"ContainerStarted","Data":"81b9dcf76a6358d970be01bcf4008caaa0c804887b7e014299c5c03826aaa11e"}
Jul 28 20:46:12.344976 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.344955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nsk8" event={"ID":"0721c49d-97a9-40c5-8742-a16ade512302","Type":"ContainerStarted","Data":"da594e6adea3e6d675e3cd1231a62f857a618ac26bf38b50fc48edd2491db4ac"}
Jul 28 20:46:12.800752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.800706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:46:12.803797 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.803768    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"image-registry-78c6bc9548-nfbrn\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") " pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:46:12.901641 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.901602    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:46:12.905189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.905162    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:12.908921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.908440    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98db2d83-d062-4737-8e53-c31ba8cb72ad-metrics-tls\") pod \"dns-default-4cfcq\" (UID: \"98db2d83-d062-4737-8e53-c31ba8cb72ad\") " pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:46:12.911118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.909502    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"]
Jul 28 20:46:12.911118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.910223    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:12.913959 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.913602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:12.917635 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.917614    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 20:46:12.918104 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.918085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 20:46:12.918354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.918330    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 20:46:12.919411 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.919381    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 20:46:12.919617 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.919593    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 20:46:12.920876 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.920855    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 20:46:12.921130 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.921113    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ps2nk\""
Jul 28 20:46:12.922985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.921405    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jfd4w\""
Jul 28 20:46:12.922985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.921593    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 20:46:12.922985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.922152    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"]
Jul 28 20:46:12.923631 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.923218    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 20:46:12.924681 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:12.924634    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/73e05204-14d9-4285-8a11-1d4e86224fd9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mz5xb\" (UID: \"73e05204-14d9-4285-8a11-1d4e86224fd9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.002634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.002537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgvj\" (UniqueName: \"kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.065395 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.065311    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kskpf\""
Jul 28 20:46:13.071549 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.071523    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:46:13.080441 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.080242    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph6jd\""
Jul 28 20:46:13.089340 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.089318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:46:13.103834 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.103942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.103942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.103942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.103942 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/73e05204-14d9-4285-8a11-1d4e86224fd9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mz5xb\" (UID: \"73e05204-14d9-4285-8a11-1d4e86224fd9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:13.104184 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.103948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.104184 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.104001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgvj\" (UniqueName: \"kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.106019 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.105393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.106019 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.105929    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.107029 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.107008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.108383 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.108363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.110178 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.110154    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.110953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.110935    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/73e05204-14d9-4285-8a11-1d4e86224fd9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mz5xb\" (UID: \"73e05204-14d9-4285-8a11-1d4e86224fd9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:13.156379 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.156330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgvj\" (UniqueName: \"kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj\") pod \"console-6c6f75fb7c-q98w9\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") " pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.231368 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.231318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:13.237472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.237402    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:13.498740 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.498711    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4cfcq"]
Jul 28 20:46:13.629622 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:13.629542    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98db2d83_d062_4737_8e53_c31ba8cb72ad.slice/crio-c0625bf584a38f99e277b5743a16b21e7522c5fe6c50ee8469c50065e4b29e90 WatchSource:0}: Error finding container c0625bf584a38f99e277b5743a16b21e7522c5fe6c50ee8469c50065e4b29e90: Status 404 returned error can't find the container with id c0625bf584a38f99e277b5743a16b21e7522c5fe6c50ee8469c50065e4b29e90
Jul 28 20:46:13.709971 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.709901    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:13.713193 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:13.713166    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b80c0f_0f2b_4001_8c09_f69db69cc292.slice/crio-2a48799e7ba6ce927a9189d8b7cfe9e74263b0682430ee108c992003f67d44f2 WatchSource:0}: Error finding container 2a48799e7ba6ce927a9189d8b7cfe9e74263b0682430ee108c992003f67d44f2: Status 404 returned error can't find the container with id 2a48799e7ba6ce927a9189d8b7cfe9e74263b0682430ee108c992003f67d44f2
Jul 28 20:46:13.716789 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.716706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:46:13.719751 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:13.719672    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"]
Jul 28 20:46:14.366191 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.365465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nsk8" event={"ID":"0721c49d-97a9-40c5-8742-a16ade512302","Type":"ContainerStarted","Data":"3350a23fdf3c795025a3143bac64426a6d6ad7ae389c53fc0c6e0111586647e7"}
Jul 28 20:46:14.366191 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.365529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nsk8" event={"ID":"0721c49d-97a9-40c5-8742-a16ade512302","Type":"ContainerStarted","Data":"8dbb3e4ba75a428022c5969e8acaf97393ba289f0f0c115acadc03554cd3b929"}
Jul 28 20:46:14.369060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.368469    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" event={"ID":"82eebc70-3a60-434f-a7bd-e7012bf0157a","Type":"ContainerStarted","Data":"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"}
Jul 28 20:46:14.369060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.368512    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" event={"ID":"82eebc70-3a60-434f-a7bd-e7012bf0157a","Type":"ContainerStarted","Data":"36a68973b939247ac93be1bca129cb63c502753be502f22ef49f319f6d76a511"}
Jul 28 20:46:14.369060 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.369004    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:46:14.370632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.370568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb" event={"ID":"73e05204-14d9-4285-8a11-1d4e86224fd9","Type":"ContainerStarted","Data":"91cdedadfdf64c7fbc2ba927bb8ba68ac91cecf093bb5a7fa537eda83df94bee"}
Jul 28 20:46:14.373317 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.373222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4cfcq" event={"ID":"98db2d83-d062-4737-8e53-c31ba8cb72ad","Type":"ContainerStarted","Data":"c0625bf584a38f99e277b5743a16b21e7522c5fe6c50ee8469c50065e4b29e90"}
Jul 28 20:46:14.377844 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.377797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6f75fb7c-q98w9" event={"ID":"55b80c0f-0f2b-4001-8c09-f69db69cc292","Type":"ContainerStarted","Data":"2a48799e7ba6ce927a9189d8b7cfe9e74263b0682430ee108c992003f67d44f2"}
Jul 28 20:46:14.425499 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.423904    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" podStartSLOduration=70.423879909 podStartE2EDuration="1m10.423879909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:14.422630347 +0000 UTC m=+68.821687012" watchObservedRunningTime="2026-07-28 20:46:14.423879909 +0000 UTC m=+68.822936575"
Jul 28 20:46:14.425499 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:14.424474    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6nsk8" podStartSLOduration=66.056447216 podStartE2EDuration="1m7.424458403s" totalImagesPullingTime="1.368011187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:14.390666207 +0000 UTC m=+68.789722883" watchObservedRunningTime="2026-07-28 20:46:14.424458403 +0000 UTC m=+68.823515067"
Jul 28 20:46:20.400654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.400602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb" event={"ID":"73e05204-14d9-4285-8a11-1d4e86224fd9","Type":"ContainerStarted","Data":"a6c1d07b6107f707b34e6638ccf3c30d9ef9e1b26b578395a81c8f840c8d6e04"}
Jul 28 20:46:20.401124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.400833    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:20.401606 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.401577    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:46:20.406528 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406508    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r27kk" event={"ID":"3512fdc5-10a8-4257-ac49-9d20b7b338c7","Type":"ContainerStarted","Data":"0324e9d0e6662600b15759f3b77b40f19723d4f2e2ba45265f9220b066eef336"}
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sqwjl" event={"ID":"b23b1762-9707-464c-a2c1-22ae6e884d18","Type":"ContainerStarted","Data":"96814a5df0541751237cb099790abe29b04f9e5fde9a122c9abe2cc8912d24de"}
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4cfcq" event={"ID":"98db2d83-d062-4737-8e53-c31ba8cb72ad","Type":"ContainerStarted","Data":"476c959a12644578930c90aeda6bac367c76fcddb164f74c5c64f899a5efc813"}
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406599    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406607    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.406632 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406622    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4cfcq" event={"ID":"98db2d83-d062-4737-8e53-c31ba8cb72ad","Type":"ContainerStarted","Data":"7133979c6e43a6b09b366cd7af15e9471a4d45242481b5fe0e942c5f31626d74"}
Jul 28 20:46:20.406884 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.406669    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb"
Jul 28 20:46:20.407555 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.407527    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6f75fb7c-q98w9" event={"ID":"55b80c0f-0f2b-4001-8c09-f69db69cc292","Type":"ContainerStarted","Data":"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"}
Jul 28 20:46:20.414022 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.414004    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 20:46:20.418265 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.418242    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:46:20.440945 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.440903    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mz5xb" podStartSLOduration=2.630984434 podStartE2EDuration="8.440886907s" totalImagesPullingTime="5.809902473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:20.420683597 +0000 UTC m=+74.819740264" watchObservedRunningTime="2026-07-28 20:46:20.440886907 +0000 UTC m=+74.839943573"
Jul 28 20:46:20.460450 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.460411    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6f75fb7c-q98w9" podStartSLOduration=2.624663208 podStartE2EDuration="8.460398878s" totalImagesPullingTime="5.83573567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:20.459809362 +0000 UTC m=+74.858866033" watchObservedRunningTime="2026-07-28 20:46:20.460398878 +0000 UTC m=+74.859455542"
Jul 28 20:46:20.460767 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.460746    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4cfcq" podStartSLOduration=34.57622751 podStartE2EDuration="40.460741137s" totalImagesPullingTime="5.884513627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:20.441212597 +0000 UTC m=+74.840269262" watchObservedRunningTime="2026-07-28 20:46:20.460741137 +0000 UTC m=+74.859797803"
Jul 28 20:46:20.462818 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.462802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463485 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463449    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463589 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbbf8\" (UniqueName: \"kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463589 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463530    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463614    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.463789 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.463689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.497356 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.497315    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r27kk" podStartSLOduration=67.234801735 podStartE2EDuration="1m14.497300244s" totalImagesPullingTime="7.262498509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:20.495931768 +0000 UTC m=+74.894988432" watchObservedRunningTime="2026-07-28 20:46:20.497300244 +0000 UTC m=+74.896356908"
Jul 28 20:46:20.533038 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.532998    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sqwjl" podStartSLOduration=66.244855718 podStartE2EDuration="1m13.532983839s" totalImagesPullingTime="7.288128121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:20.531845833 +0000 UTC m=+74.930902497" watchObservedRunningTime="2026-07-28 20:46:20.532983839 +0000 UTC m=+74.932040504"
Jul 28 20:46:20.564888 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.564866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:20.564994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.564895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.564994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.564912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.564994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.564936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.564994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.564972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.565445 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.565399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.565622 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.565598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbbf8\" (UniqueName: \"kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.565751 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.565733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.566151 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.566097    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.566713 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.566688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.566870 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.566719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.567086 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.567061    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.568506 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.568478    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.568889 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.568865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.571052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.571032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514b3bd1-2f12-4d4a-b437-63d4e629fcfd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ddf7\" (UID: \"514b3bd1-2f12-4d4a-b437-63d4e629fcfd\") " pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:20.577283 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.577259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbbf8\" (UniqueName: \"kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8\") pod \"console-58cd7d878-snbqk\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") " pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.715602 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.715526    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:20.844558 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.844529    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:46:20.847473 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:20.847436    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5066030c_0e53_44f2_88f9_1c76d5125ad9.slice/crio-f80e13809288ddc9cb13ce18291746975384dd48dc640f689b3e8bad015c3e7b WatchSource:0}: Error finding container f80e13809288ddc9cb13ce18291746975384dd48dc640f689b3e8bad015c3e7b: Status 404 returned error can't find the container with id f80e13809288ddc9cb13ce18291746975384dd48dc640f689b3e8bad015c3e7b
Jul 28 20:46:20.853284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.853268    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nwhvj\""
Jul 28 20:46:20.861518 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.861502    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5ddf7"
Jul 28 20:46:20.990871 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:20.990798    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5ddf7"]
Jul 28 20:46:20.993835 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:20.993809    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514b3bd1_2f12_4d4a_b437_63d4e629fcfd.slice/crio-5a6f687d5cf0d9dc4de401eab2f3d4f13e86cbc6ce8d4a000b324cc5069b7a2b WatchSource:0}: Error finding container 5a6f687d5cf0d9dc4de401eab2f3d4f13e86cbc6ce8d4a000b324cc5069b7a2b: Status 404 returned error can't find the container with id 5a6f687d5cf0d9dc4de401eab2f3d4f13e86cbc6ce8d4a000b324cc5069b7a2b
Jul 28 20:46:21.416012 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.414370    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ddf7" event={"ID":"514b3bd1-2f12-4d4a-b437-63d4e629fcfd","Type":"ContainerStarted","Data":"5a6f687d5cf0d9dc4de401eab2f3d4f13e86cbc6ce8d4a000b324cc5069b7a2b"}
Jul 28 20:46:21.416690 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.416661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cd7d878-snbqk" event={"ID":"5066030c-0e53-44f2-88f9-1c76d5125ad9","Type":"ContainerStarted","Data":"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"}
Jul 28 20:46:21.416774 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.416707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cd7d878-snbqk" event={"ID":"5066030c-0e53-44f2-88f9-1c76d5125ad9","Type":"ContainerStarted","Data":"f80e13809288ddc9cb13ce18291746975384dd48dc640f689b3e8bad015c3e7b"}
Jul 28 20:46:21.438829 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.438772    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58cd7d878-snbqk" podStartSLOduration=1.4387521429999999 podStartE2EDuration="1.438752143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:21.437417152 +0000 UTC m=+75.836473817" watchObservedRunningTime="2026-07-28 20:46:21.438752143 +0000 UTC m=+75.837808810"
Jul 28 20:46:21.558865 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.558075    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"]
Jul 28 20:46:21.561260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.561236    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.566164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.565741    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 20:46:21.566164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.565766    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 20:46:21.566164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.566090    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 20:46:21.566460 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.566435    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 20:46:21.566780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.566760    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 20:46:21.566969 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.566951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2zq56\""
Jul 28 20:46:21.578980 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.578950    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"]
Jul 28 20:46:21.678397 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.678290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495e1672-c7fe-4d51-921c-d44eee1f1b50-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.678397 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.678332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.678397 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.678369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f469\" (UniqueName: \"kubernetes.io/projected/495e1672-c7fe-4d51-921c-d44eee1f1b50-kube-api-access-2f469\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.678662 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.678421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.781013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.779351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.781013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.779411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495e1672-c7fe-4d51-921c-d44eee1f1b50-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.781013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.779439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.781013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.779498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f469\" (UniqueName: \"kubernetes.io/projected/495e1672-c7fe-4d51-921c-d44eee1f1b50-kube-api-access-2f469\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.781013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.780962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495e1672-c7fe-4d51-921c-d44eee1f1b50-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.790074 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.790041    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.797161 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.797119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495e1672-c7fe-4d51-921c-d44eee1f1b50-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.799926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.799905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f469\" (UniqueName: \"kubernetes.io/projected/495e1672-c7fe-4d51-921c-d44eee1f1b50-kube-api-access-2f469\") pod \"prometheus-operator-84d69f794d-vbrf4\" (UID: \"495e1672-c7fe-4d51-921c-d44eee1f1b50\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:21.889039 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:21.889007    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"
Jul 28 20:46:22.040268 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:22.040234    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vbrf4"]
Jul 28 20:46:22.044256 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:22.044225    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495e1672_c7fe_4d51_921c_d44eee1f1b50.slice/crio-8d359e20fd61068e8db2d80fd6659359d954eb418b525a1725ce01ca5deadc5d WatchSource:0}: Error finding container 8d359e20fd61068e8db2d80fd6659359d954eb418b525a1725ce01ca5deadc5d: Status 404 returned error can't find the container with id 8d359e20fd61068e8db2d80fd6659359d954eb418b525a1725ce01ca5deadc5d
Jul 28 20:46:22.423761 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:22.423730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4" event={"ID":"495e1672-c7fe-4d51-921c-d44eee1f1b50","Type":"ContainerStarted","Data":"8d359e20fd61068e8db2d80fd6659359d954eb418b525a1725ce01ca5deadc5d"}
Jul 28 20:46:23.232499 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:23.232462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:23.232682 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:23.232540    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:23.238926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:23.238892    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:23.431407 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:23.431380    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:26.438650 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:26.438573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ddf7" event={"ID":"514b3bd1-2f12-4d4a-b437-63d4e629fcfd","Type":"ContainerStarted","Data":"862d6d5f3c871c9596f14ef8f4222cc558b8bbea496dae3ad068ae9747ebe473"}
Jul 28 20:46:26.440276 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:26.440252    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4" event={"ID":"495e1672-c7fe-4d51-921c-d44eee1f1b50","Type":"ContainerStarted","Data":"29644ff6272593afb21902de205bcc8c7de6db1727dc6c3880aa0b31b78cb2f5"}
Jul 28 20:46:26.440386 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:26.440296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4" event={"ID":"495e1672-c7fe-4d51-921c-d44eee1f1b50","Type":"ContainerStarted","Data":"2ef797f23dac6518bbeca1576b6b560b71ae12e301c02fb267dd78be610772cc"}
Jul 28 20:46:26.458488 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:26.458449    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-vbrf4" podStartSLOduration=1.473716908 podStartE2EDuration="5.45843414s" totalImagesPullingTime="3.984717232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:26.457378686 +0000 UTC m=+80.856435354" watchObservedRunningTime="2026-07-28 20:46:26.45843414 +0000 UTC m=+80.857490804"
Jul 28 20:46:27.965377 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.965257    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"]
Jul 28 20:46:27.975351 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.975331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:27.978654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.978632    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 20:46:27.978809 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.978637    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 20:46:27.978809 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.978718    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q4d4b\""
Jul 28 20:46:27.979917 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.979895    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 20:46:27.980426 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.980409    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fj8lf"]
Jul 28 20:46:27.983909 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.983887    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"]
Jul 28 20:46:27.984012 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.983973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:27.986028 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.986008    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 20:46:27.986362 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.986340    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 20:46:27.986450 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.986432    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tsvt2\""
Jul 28 20:46:27.986986 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:27.986965    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 20:46:28.129077 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.129276 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-metrics-client-ca\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129276 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-root\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129276 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-wtmp\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129276 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129334    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45988f53-82ef-4c7a-8d00-edf881f30ce2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.129472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-textfile\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129472 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxpj2\" (UniqueName: \"kubernetes.io/projected/c7d3ad98-23b1-4bda-a877-10b3845af57d-kube-api-access-zxpj2\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-tls\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129575    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.129654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.129770 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-sys\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.129770 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvck8\" (UniqueName: \"kubernetes.io/projected/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-api-access-pvck8\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.129770 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.129721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.230863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxpj2\" (UniqueName: \"kubernetes.io/projected/c7d3ad98-23b1-4bda-a877-10b3845af57d-kube-api-access-zxpj2\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.230911    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-tls\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.230942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.230973    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.230991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-sys\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.231168 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.231018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvck8\" (UniqueName: \"kubernetes.io/projected/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-api-access-pvck8\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.232377 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-sys\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232544    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-metrics-client-ca\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-root\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-wtmp\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45988f53-82ef-4c7a-8d00-edf881f30ce2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-textfile\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.233346    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-metrics-client-ca\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.233746 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:28.233697    2567 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 20:46:28.233796 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:28.233756    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls podName:45988f53-82ef-4c7a-8d00-edf881f30ce2 nodeName:}" failed. No retries permitted until 2026-07-28 20:46:28.733737191 +0000 UTC m=+83.132793837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-rvxwz" (UID: "45988f53-82ef-4c7a-8d00-edf881f30ce2") : secret "kube-state-metrics-tls" not found
Jul 28 20:46:28.233887 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.233857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.234013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.233944    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-root\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.234013 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.232773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.234229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.234210    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-textfile\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.234496 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.234450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45988f53-82ef-4c7a-8d00-edf881f30ce2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.234621 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.234604    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-wtmp\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.234808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.234760    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45988f53-82ef-4c7a-8d00-edf881f30ce2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.235637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.235612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.235730 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.235618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.237872 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.237851    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7d3ad98-23b1-4bda-a877-10b3845af57d-node-exporter-tls\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.246266 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.246236    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxpj2\" (UniqueName: \"kubernetes.io/projected/c7d3ad98-23b1-4bda-a877-10b3845af57d-kube-api-access-zxpj2\") pod \"node-exporter-fj8lf\" (UID: \"c7d3ad98-23b1-4bda-a877-10b3845af57d\") " pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.247864 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.247823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvck8\" (UniqueName: \"kubernetes.io/projected/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-api-access-pvck8\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.296867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.296833    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj8lf"
Jul 28 20:46:28.736852 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.736824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.739310 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.739288    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45988f53-82ef-4c7a-8d00-edf881f30ce2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rvxwz\" (UID: \"45988f53-82ef-4c7a-8d00-edf881f30ce2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:28.887027 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:28.886997    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"
Jul 28 20:46:29.483297 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:29.483074    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d3ad98_23b1_4bda_a877_10b3845af57d.slice/crio-f5833daa2ea279c5709512f2caafbd4f4b329533e0fc6c8aa98f7658d4cdf15f WatchSource:0}: Error finding container f5833daa2ea279c5709512f2caafbd4f4b329533e0fc6c8aa98f7658d4cdf15f: Status 404 returned error can't find the container with id f5833daa2ea279c5709512f2caafbd4f4b329533e0fc6c8aa98f7658d4cdf15f
Jul 28 20:46:29.614806 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:29.614775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz"]
Jul 28 20:46:29.617265 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:29.617240    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45988f53_82ef_4c7a_8d00_edf881f30ce2.slice/crio-4f5c357049a068ceb50a582f80b6a10d769879f890cdb6b34186c8d2952e3acc WatchSource:0}: Error finding container 4f5c357049a068ceb50a582f80b6a10d769879f890cdb6b34186c8d2952e3acc: Status 404 returned error can't find the container with id 4f5c357049a068ceb50a582f80b6a10d769879f890cdb6b34186c8d2952e3acc
Jul 28 20:46:30.021343 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.021307    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"]
Jul 28 20:46:30.024983 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.024961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.027954 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.027352    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fcgtb\""
Jul 28 20:46:30.027954 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.027392    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 20:46:30.027954 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.027707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cv8sn04l3ac8u\""
Jul 28 20:46:30.027954 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.027718    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 20:46:30.027954 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.027854    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 20:46:30.028324 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.028097    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 20:46:30.028324 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.028105    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 20:46:30.038589 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.038570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"]
Jul 28 20:46:30.148642 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148594    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.148807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56d3951-e4ef-4091-af09-961b4837a911-metrics-client-ca\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.148807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.148807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148723    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.148807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148753    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-grpc-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.149020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.149020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8pj\" (UniqueName: \"kubernetes.io/projected/f56d3951-e4ef-4091-af09-961b4837a911-kube-api-access-hw8pj\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.149020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.148910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.249989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56d3951-e4ef-4091-af09-961b4837a911-metrics-client-ca\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-grpc-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8pj\" (UniqueName: \"kubernetes.io/projected/f56d3951-e4ef-4091-af09-961b4837a911-kube-api-access-hw8pj\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.250301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.250253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.251825 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.251798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56d3951-e4ef-4091-af09-961b4837a911-metrics-client-ca\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.253313 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.253286    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.253740 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.253705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.253834 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.253809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.254830 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.254786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.255403 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.255380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.255824 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.255785    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f56d3951-e4ef-4091-af09-961b4837a911-secret-grpc-tls\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.259957 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.259936    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8pj\" (UniqueName: \"kubernetes.io/projected/f56d3951-e4ef-4091-af09-961b4837a911-kube-api-access-hw8pj\") pod \"thanos-querier-84bc9f8bb7-scjpc\" (UID: \"f56d3951-e4ef-4091-af09-961b4837a911\") " pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.336351 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.336283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:30.419241 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.419188    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4cfcq"
Jul 28 20:46:30.463684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.460522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj8lf" event={"ID":"c7d3ad98-23b1-4bda-a877-10b3845af57d","Type":"ContainerStarted","Data":"f5833daa2ea279c5709512f2caafbd4f4b329533e0fc6c8aa98f7658d4cdf15f"}
Jul 28 20:46:30.463684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.462012    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz" event={"ID":"45988f53-82ef-4c7a-8d00-edf881f30ce2","Type":"ContainerStarted","Data":"4f5c357049a068ceb50a582f80b6a10d769879f890cdb6b34186c8d2952e3acc"}
Jul 28 20:46:30.464790 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.464711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ddf7" event={"ID":"514b3bd1-2f12-4d4a-b437-63d4e629fcfd","Type":"ContainerStarted","Data":"0bac0d8d86735c7b8d00bf648e2d2cba28b33781fc7cdc857326d5bbef1cb5ca"}
Jul 28 20:46:30.509840 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.509742    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"]
Jul 28 20:46:30.706300 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:30.706253    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56d3951_e4ef_4091_af09_961b4837a911.slice/crio-f360757e97b2af0968c1ad8a84baf04a162d6cf155c4e4a3dfb550373809f5a9 WatchSource:0}: Error finding container f360757e97b2af0968c1ad8a84baf04a162d6cf155c4e4a3dfb550373809f5a9: Status 404 returned error can't find the container with id f360757e97b2af0968c1ad8a84baf04a162d6cf155c4e4a3dfb550373809f5a9
Jul 28 20:46:30.717188 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.716563    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:30.717341 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.717322    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:30.721945 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:30.721927    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:31.473401 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.473298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"f360757e97b2af0968c1ad8a84baf04a162d6cf155c4e4a3dfb550373809f5a9"}
Jul 28 20:46:31.478792 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.478762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz" event={"ID":"45988f53-82ef-4c7a-8d00-edf881f30ce2","Type":"ContainerStarted","Data":"8318fc50cac9d07b9c95d091485d7f771d3ef3033d09c7c0be1f52ce8233499c"}
Jul 28 20:46:31.478917 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.478814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz" event={"ID":"45988f53-82ef-4c7a-8d00-edf881f30ce2","Type":"ContainerStarted","Data":"dac85799cdf64c6f3299b50bdaefe73cb43d53435dfcb99add0607832ab41a87"}
Jul 28 20:46:31.481527 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.480706    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7d3ad98-23b1-4bda-a877-10b3845af57d" containerID="959301f985300b0c27e384f95ced8a76ee03582d5d11eda1b97a27323f8f2698" exitCode=0
Jul 28 20:46:31.481783 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.481753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj8lf" event={"ID":"c7d3ad98-23b1-4bda-a877-10b3845af57d","Type":"ContainerDied","Data":"959301f985300b0c27e384f95ced8a76ee03582d5d11eda1b97a27323f8f2698"}
Jul 28 20:46:31.489229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.488782    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:46:31.561985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:31.561960    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:32.487592 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.487557    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz" event={"ID":"45988f53-82ef-4c7a-8d00-edf881f30ce2","Type":"ContainerStarted","Data":"46c4e05b09e34afbc7792ae04985d81a307ba5fe63603d2ff7a1185532a2fc84"}
Jul 28 20:46:32.490011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.489987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj8lf" event={"ID":"c7d3ad98-23b1-4bda-a877-10b3845af57d","Type":"ContainerStarted","Data":"0d22a759cb607138243649de638436354f1cc763d7a2920852c199b301393452"}
Jul 28 20:46:32.490161 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.490030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj8lf" event={"ID":"c7d3ad98-23b1-4bda-a877-10b3845af57d","Type":"ContainerStarted","Data":"9ce6b026c389fa8768ff4926d244113772935fcb74e195b436b20b993cbd9522"}
Jul 28 20:46:32.511780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.510882    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rvxwz" podStartSLOduration=3.811906064 podStartE2EDuration="5.510862926s" totalImagesPullingTime="1.698956862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:32.510776825 +0000 UTC m=+86.909833493" watchObservedRunningTime="2026-07-28 20:46:32.510862926 +0000 UTC m=+86.909919590"
Jul 28 20:46:32.532799 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.532755    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fj8lf" podStartSLOduration=4.172131566 podStartE2EDuration="5.532740299s" totalImagesPullingTime="1.269034494s" totalInitContainerRuntime="91.574239ms" isStatefulPod=true podCreationTimestamp="2026-07-28 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:32.531013668 +0000 UTC m=+86.930070333" watchObservedRunningTime="2026-07-28 20:46:32.532740299 +0000 UTC m=+86.931796965"
Jul 28 20:46:32.711000 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.709878    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"]
Jul 28 20:46:32.715625 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.715597    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:32.718108 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.718083    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 20:46:32.718108 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.718099    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tp98z\""
Jul 28 20:46:32.724916 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.724888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"]
Jul 28 20:46:32.878381 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.878345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hqh\" (UniqueName: \"kubernetes.io/projected/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-kube-api-access-88hqh\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:32.878551 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.878403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:32.978947 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.978916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hqh\" (UniqueName: \"kubernetes.io/projected/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-kube-api-access-88hqh\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:32.979087 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.978972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:32.979163 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:32.979085    2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 20:46:32.979222 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:32.979170    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert podName:95b7ef76-3e7e-4db2-b19f-7775fe07b4d6 nodeName:}" failed. No retries permitted until 2026-07-28 20:46:33.479131245 +0000 UTC m=+87.878187902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-rb94r" (UID: "95b7ef76-3e7e-4db2-b19f-7775fe07b4d6") : secret "monitoring-plugin-cert" not found
Jul 28 20:46:32.988376 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:32.988345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hqh\" (UniqueName: \"kubernetes.io/projected/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-kube-api-access-88hqh\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:33.483743 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:33.483712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:33.486109 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:33.486086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95b7ef76-3e7e-4db2-b19f-7775fe07b4d6-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rb94r\" (UID: \"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:33.629482 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:33.629451    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:34.236170 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.236086    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"]
Jul 28 20:46:34.240435 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:34.240408    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b7ef76_3e7e_4db2_b19f_7775fe07b4d6.slice/crio-861b1038a4597be41c00739cbd24684ca33eec4b7314a2788b9c6fca3a293a7a WatchSource:0}: Error finding container 861b1038a4597be41c00739cbd24684ca33eec4b7314a2788b9c6fca3a293a7a: Status 404 returned error can't find the container with id 861b1038a4597be41c00739cbd24684ca33eec4b7314a2788b9c6fca3a293a7a
Jul 28 20:46:34.501944 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.501683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ddf7" event={"ID":"514b3bd1-2f12-4d4a-b437-63d4e629fcfd","Type":"ContainerStarted","Data":"ae9d99bb1619133a1ff18d9dc17b4af9abc7a6e4acfab5087cc25bc188c269a9"}
Jul 28 20:46:34.504471 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.504432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"dfc4f62fde036812fdc1f41e90873be75c7ae1827ea69a12c0fac6a1d051bb33"}
Jul 28 20:46:34.504670 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.504516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"d8d3f9eef5492d37a71222230169836bd4ab7704fe1735dc59b922af0580c239"}
Jul 28 20:46:34.504670 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.504560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"f195cf40212bd52931a008e8bda8a90cf03b66c28ca2236aba70579266e493b0"}
Jul 28 20:46:34.505714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.505687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r" event={"ID":"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6","Type":"ContainerStarted","Data":"861b1038a4597be41c00739cbd24684ca33eec4b7314a2788b9c6fca3a293a7a"}
Jul 28 20:46:34.526907 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.526842    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5ddf7" podStartSLOduration=33.57355035 podStartE2EDuration="46.526819837s" totalImagesPullingTime="12.953269487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:45:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:34.525117312 +0000 UTC m=+88.924173981" watchObservedRunningTime="2026-07-28 20:46:34.526819837 +0000 UTC m=+88.925876501"
Jul 28 20:46:34.831100 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.831070    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:46:34.834258 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.834232    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.847337 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.847316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:46:34.896521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.896490    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:46:34.902387 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.902365    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:46:34.997756 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.997709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.997767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.997798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmxl2\" (UniqueName: \"kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998162 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.998050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998162 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.998092    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998269 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.998177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:34.998269 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:34.998210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099519 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099519 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.099725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.099672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmxl2\" (UniqueName: \"kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.100928 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.100901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.101053 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.100903    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.101109 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.101088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.101264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.101242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.102442 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.102425    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.102847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.102827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.117888 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.117862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmxl2\" (UniqueName: \"kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2\") pod \"console-9fdcff5db-nnphp\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") " pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.143461 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.143272    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:35.399075 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.399033    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:46:35.513673 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:35.513631    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"eb94e3363c94595158f115f4ea44734157c4c12a39f79fd67f0d0604014f6043"}
Jul 28 20:46:35.727453 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:46:35.727420    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a34582e_fb12_45f6_b6b1_4601327e65e0.slice/crio-7ebed4d8b6e8d51aaccb3fdb5e89f1103bddddddb24b916863989427e6f91e33 WatchSource:0}: Error finding container 7ebed4d8b6e8d51aaccb3fdb5e89f1103bddddddb24b916863989427e6f91e33: Status 404 returned error can't find the container with id 7ebed4d8b6e8d51aaccb3fdb5e89f1103bddddddb24b916863989427e6f91e33
Jul 28 20:46:36.521497 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.521455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"93de654a973e73e89c5d90cadf3b5e86e9e4529ef8f7e0c4ebb25a5d6cc99c9f"}
Jul 28 20:46:36.521926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.521540    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" event={"ID":"f56d3951-e4ef-4091-af09-961b4837a911","Type":"ContainerStarted","Data":"4050163f5860ac6417315e920aaa7765fecefdb5ea07a6bb8b532f5b4552f411"}
Jul 28 20:46:36.521926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.521592    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:36.522882 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.522857    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r" event={"ID":"95b7ef76-3e7e-4db2-b19f-7775fe07b4d6","Type":"ContainerStarted","Data":"46e35bd7c47542d8272c19bf2d6cae68aa7cb19d0ac85a13977e8c991260b0bd"}
Jul 28 20:46:36.523041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.523010    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:36.524583 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.524555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdcff5db-nnphp" event={"ID":"0a34582e-fb12-45f6-b6b1-4601327e65e0","Type":"ContainerStarted","Data":"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"}
Jul 28 20:46:36.524696 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.524598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdcff5db-nnphp" event={"ID":"0a34582e-fb12-45f6-b6b1-4601327e65e0","Type":"ContainerStarted","Data":"7ebed4d8b6e8d51aaccb3fdb5e89f1103bddddddb24b916863989427e6f91e33"}
Jul 28 20:46:36.528975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.528956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r"
Jul 28 20:46:36.549073 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.549018    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc" podStartSLOduration=3.1979034139999998 podStartE2EDuration="7.549001963s" totalImagesPullingTime="4.351098549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:36.547916894 +0000 UTC m=+90.946973569" watchObservedRunningTime="2026-07-28 20:46:36.549001963 +0000 UTC m=+90.948058673"
Jul 28 20:46:36.567734 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.567651    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9fdcff5db-nnphp" podStartSLOduration=2.567634285 podStartE2EDuration="2.567634285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:36.567462528 +0000 UTC m=+90.966519193" watchObservedRunningTime="2026-07-28 20:46:36.567634285 +0000 UTC m=+90.966690948"
Jul 28 20:46:36.584492 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:36.584446    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rb94r" podStartSLOduration=3.050422944 podStartE2EDuration="4.584426011s" totalImagesPullingTime="1.534003067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:36.583291233 +0000 UTC m=+90.982347909" watchObservedRunningTime="2026-07-28 20:46:36.584426011 +0000 UTC m=+90.983482678"
Jul 28 20:46:42.534202 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:42.534172    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84bc9f8bb7-scjpc"
Jul 28 20:46:45.143504 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:45.143456    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:45.143959 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:45.143536    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:45.148066 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:45.148044    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:45.564438 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:45.564412    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:46:45.621578 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:45.621550    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:46:51.419028 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:51.418997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r27kk"
Jul 28 20:46:56.592810 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.592767    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c6f75fb7c-q98w9" podUID="55b80c0f-0f2b-4001-8c09-f69db69cc292" containerName="console" containerID="cri-o://e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f" gracePeriod=15
Jul 28 20:46:56.878782 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.878756    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:56.977751 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977716    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.977926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977778    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.977926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977881    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.977926 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977923    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jgvj\" (UniqueName: \"kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.978052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.978052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.977985    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert\") pod \"55b80c0f-0f2b-4001-8c09-f69db69cc292\" (UID: \"55b80c0f-0f2b-4001-8c09-f69db69cc292\") "
Jul 28 20:46:56.978240 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.978216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:46:56.978317 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.978221    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca" (OuterVolumeSpecName: "service-ca") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:46:56.978317 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.978254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config" (OuterVolumeSpecName: "console-config") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:46:56.980534 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.980508    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:46:56.980634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.980594    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:46:56.980634 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:56.980611    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj" (OuterVolumeSpecName: "kube-api-access-9jgvj") pod "55b80c0f-0f2b-4001-8c09-f69db69cc292" (UID: "55b80c0f-0f2b-4001-8c09-f69db69cc292"). InnerVolumeSpecName "kube-api-access-9jgvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:46:57.078781 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078745    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jgvj\" (UniqueName: \"kubernetes.io/projected/55b80c0f-0f2b-4001-8c09-f69db69cc292-kube-api-access-9jgvj\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.078781 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078775    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-oauth-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.078781 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078784    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.078781 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078793    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-oauth-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.079072 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078802    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.079072 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.078810    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55b80c0f-0f2b-4001-8c09-f69db69cc292-console-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:46:57.606443 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.606414    2567 generic.go:356] "Generic (PLEG): container finished" podID="55b80c0f-0f2b-4001-8c09-f69db69cc292" containerID="e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f" exitCode=0
Jul 28 20:46:57.606947 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.606472    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6f75fb7c-q98w9" event={"ID":"55b80c0f-0f2b-4001-8c09-f69db69cc292","Type":"ContainerDied","Data":"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"}
Jul 28 20:46:57.606947 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.606474    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6f75fb7c-q98w9"
Jul 28 20:46:57.606947 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.606512    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6f75fb7c-q98w9" event={"ID":"55b80c0f-0f2b-4001-8c09-f69db69cc292","Type":"ContainerDied","Data":"2a48799e7ba6ce927a9189d8b7cfe9e74263b0682430ee108c992003f67d44f2"}
Jul 28 20:46:57.606947 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.606532    2567 scope.go:117] "RemoveContainer" containerID="e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"
Jul 28 20:46:57.615364 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.615348    2567 scope.go:117] "RemoveContainer" containerID="e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"
Jul 28 20:46:57.615639 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:46:57.615621    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f\": container with ID starting with e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f not found: ID does not exist" containerID="e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"
Jul 28 20:46:57.615688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.615650    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f"} err="failed to get container status \"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f\": rpc error: code = NotFound desc = could not find container \"e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f\": container with ID starting with e511d31d9f1ff885a2f047814a2e27fd5aa6f018d6507004f3cae57a1295369f not found: ID does not exist"
Jul 28 20:46:57.626314 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.626290    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:57.630590 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:57.630572    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c6f75fb7c-q98w9"]
Jul 28 20:46:58.074523 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:58.074483    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b80c0f-0f2b-4001-8c09-f69db69cc292" path="/var/lib/kubelet/pods/55b80c0f-0f2b-4001-8c09-f69db69cc292/volumes"
Jul 28 20:46:59.919197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:46:59.919152    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" podUID="82eebc70-3a60-434f-a7bd-e7012bf0157a" containerName="registry" containerID="cri-o://85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3" gracePeriod=30
Jul 28 20:47:00.159658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.159630    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:47:00.202752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202676    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202752 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202765    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qghfm\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202803    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202819    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202843    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.202970 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.202914    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") pod \"82eebc70-3a60-434f-a7bd-e7012bf0157a\" (UID: \"82eebc70-3a60-434f-a7bd-e7012bf0157a\") "
Jul 28 20:47:00.203315 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.203286    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:00.203874 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.203842    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:00.205544 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.205443    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:00.205544 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.205486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:00.205714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.205577    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm" (OuterVolumeSpecName: "kube-api-access-qghfm") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "kube-api-access-qghfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:00.205714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.205622    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:00.205714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.205655    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:00.214331 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.214304    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "82eebc70-3a60-434f-a7bd-e7012bf0157a" (UID: "82eebc70-3a60-434f-a7bd-e7012bf0157a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:00.304197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304120    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-certificates\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304191    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-registry-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304202    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-installation-pull-secrets\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304210    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-bound-sa-token\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304219    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qghfm\" (UniqueName: \"kubernetes.io/projected/82eebc70-3a60-434f-a7bd-e7012bf0157a-kube-api-access-qghfm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304495 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304228    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82eebc70-3a60-434f-a7bd-e7012bf0157a-ca-trust-extracted\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304495 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304236    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82eebc70-3a60-434f-a7bd-e7012bf0157a-trusted-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.304495 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.304246    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82eebc70-3a60-434f-a7bd-e7012bf0157a-image-registry-private-configuration\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:00.618936 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.618905    2567 generic.go:356] "Generic (PLEG): container finished" podID="82eebc70-3a60-434f-a7bd-e7012bf0157a" containerID="85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3" exitCode=0
Jul 28 20:47:00.619126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.618951    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn"
Jul 28 20:47:00.619126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.618996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" event={"ID":"82eebc70-3a60-434f-a7bd-e7012bf0157a","Type":"ContainerDied","Data":"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"}
Jul 28 20:47:00.619126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.619047    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c6bc9548-nfbrn" event={"ID":"82eebc70-3a60-434f-a7bd-e7012bf0157a","Type":"ContainerDied","Data":"36a68973b939247ac93be1bca129cb63c502753be502f22ef49f319f6d76a511"}
Jul 28 20:47:00.619126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.619074    2567 scope.go:117] "RemoveContainer" containerID="85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"
Jul 28 20:47:00.628026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.627999    2567 scope.go:117] "RemoveContainer" containerID="85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"
Jul 28 20:47:00.628304 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:47:00.628284    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3\": container with ID starting with 85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3 not found: ID does not exist" containerID="85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"
Jul 28 20:47:00.628377 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.628317    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3"} err="failed to get container status \"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3\": rpc error: code = NotFound desc = could not find container \"85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3\": container with ID starting with 85c98da11e0df9bcf46a7103757742a24d6d9543ed3c0ff3753fa5075cebffd3 not found: ID does not exist"
Jul 28 20:47:00.641271 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.641245    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:47:00.645471 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:00.645450    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78c6bc9548-nfbrn"]
Jul 28 20:47:02.074788 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:02.074759    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82eebc70-3a60-434f-a7bd-e7012bf0157a" path="/var/lib/kubelet/pods/82eebc70-3a60-434f-a7bd-e7012bf0157a/volumes"
Jul 28 20:47:10.641062 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.640996    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58cd7d878-snbqk" podUID="5066030c-0e53-44f2-88f9-1c76d5125ad9" containerName="console" containerID="cri-o://2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0" gracePeriod=15
Jul 28 20:47:10.881540 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.881517    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:47:10.981737 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbbf8\" (UniqueName: \"kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981737 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981696    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981804    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981838    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981873    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981898    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.981953 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.981917    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert\") pod \"5066030c-0e53-44f2-88f9-1c76d5125ad9\" (UID: \"5066030c-0e53-44f2-88f9-1c76d5125ad9\") "
Jul 28 20:47:10.982230 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.982171    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config" (OuterVolumeSpecName: "console-config") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:10.982230 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.982186    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca" (OuterVolumeSpecName: "service-ca") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:10.982338 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.982222    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:10.982447 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.982425    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:47:10.984020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.983994    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8" (OuterVolumeSpecName: "kube-api-access-hbbf8") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "kube-api-access-hbbf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:10.984020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.984002    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:10.984278 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:10.984263    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5066030c-0e53-44f2-88f9-1c76d5125ad9" (UID: "5066030c-0e53-44f2-88f9-1c76d5125ad9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:11.082443 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082405    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbbf8\" (UniqueName: \"kubernetes.io/projected/5066030c-0e53-44f2-88f9-1c76d5125ad9-kube-api-access-hbbf8\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082443 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082436    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082443 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082446    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082453    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-trusted-ca-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082462    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-oauth-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082470    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5066030c-0e53-44f2-88f9-1c76d5125ad9-console-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.082637 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.082478    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5066030c-0e53-44f2-88f9-1c76d5125ad9-oauth-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:47:11.661455 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.661422    2567 generic.go:356] "Generic (PLEG): container finished" podID="5066030c-0e53-44f2-88f9-1c76d5125ad9" containerID="2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0" exitCode=0
Jul 28 20:47:11.661914 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.661468    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cd7d878-snbqk"
Jul 28 20:47:11.661914 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.661515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cd7d878-snbqk" event={"ID":"5066030c-0e53-44f2-88f9-1c76d5125ad9","Type":"ContainerDied","Data":"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"}
Jul 28 20:47:11.661914 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.661574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cd7d878-snbqk" event={"ID":"5066030c-0e53-44f2-88f9-1c76d5125ad9","Type":"ContainerDied","Data":"f80e13809288ddc9cb13ce18291746975384dd48dc640f689b3e8bad015c3e7b"}
Jul 28 20:47:11.661914 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.661607    2567 scope.go:117] "RemoveContainer" containerID="2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"
Jul 28 20:47:11.670405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.670372    2567 scope.go:117] "RemoveContainer" containerID="2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"
Jul 28 20:47:11.670632 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:47:11.670613    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0\": container with ID starting with 2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0 not found: ID does not exist" containerID="2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"
Jul 28 20:47:11.670684 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.670641    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0"} err="failed to get container status \"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0\": rpc error: code = NotFound desc = could not find container \"2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0\": container with ID starting with 2df41d62651c0268c2855bdfdbcda3a58cf4baa8faaac2283eab557e24de15f0 not found: ID does not exist"
Jul 28 20:47:11.682071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.682050    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:47:11.685760 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:11.685741    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58cd7d878-snbqk"]
Jul 28 20:47:12.075722 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:12.075691    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5066030c-0e53-44f2-88f9-1c76d5125ad9" path="/var/lib/kubelet/pods/5066030c-0e53-44f2-88f9-1c76d5125ad9/volumes"
Jul 28 20:47:50.305808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.305778    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306045    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82eebc70-3a60-434f-a7bd-e7012bf0157a" containerName="registry"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306056    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82eebc70-3a60-434f-a7bd-e7012bf0157a" containerName="registry"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306070    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b80c0f-0f2b-4001-8c09-f69db69cc292" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306075    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b80c0f-0f2b-4001-8c09-f69db69cc292" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306090    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5066030c-0e53-44f2-88f9-1c76d5125ad9" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306095    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5066030c-0e53-44f2-88f9-1c76d5125ad9" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306153    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5066030c-0e53-44f2-88f9-1c76d5125ad9" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306161    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b80c0f-0f2b-4001-8c09-f69db69cc292" containerName="console"
Jul 28 20:47:50.306277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.306170    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82eebc70-3a60-434f-a7bd-e7012bf0157a" containerName="registry"
Jul 28 20:47:50.309595 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.309571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.311964 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.311940    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 20:47:50.312118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.311940    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 20:47:50.312118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.311942    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 20:47:50.312118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.312114    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 20:47:50.312324 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.311950    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 20:47:50.312324 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.311948    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 20:47:50.312484 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.312396    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n9sc4\""
Jul 28 20:47:50.312731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.312716    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 20:47:50.312813 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.312717    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 20:47:50.330835 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.330816    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 20:47:50.381966 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.381935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382097 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.381979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382097 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382097 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382284 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq7p8\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-kube-api-access-sq7p8\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382595 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382318    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.382595 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.382343    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.427446 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.427422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:47:50.483082 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483481 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483546 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483546 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.483658 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.483655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq7p8\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-kube-api-access-sq7p8\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.484446 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.484077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.484556 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.484471    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.484556 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.484446    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.486863 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.486828    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487235 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.486956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487235 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487361 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487217    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487361 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487361 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487511 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.487708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.487686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.488543 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.488519    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.492526 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.492508    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq7p8\" (UniqueName: \"kubernetes.io/projected/8cc04b02-32c0-4e69-b872-bb97fc3b58cf-kube-api-access-sq7p8\") pod \"alertmanager-main-0\" (UID: \"8cc04b02-32c0-4e69-b872-bb97fc3b58cf\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.618302 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.618230    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:47:50.758805 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.758700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:47:50.765474 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:47:50.765449    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc04b02_32c0_4e69_b872_bb97fc3b58cf.slice/crio-dd6ab2202a2073879aee143e1665e1371aa16304caaf1f680c12a69b10eb3760 WatchSource:0}: Error finding container dd6ab2202a2073879aee143e1665e1371aa16304caaf1f680c12a69b10eb3760: Status 404 returned error can't find the container with id dd6ab2202a2073879aee143e1665e1371aa16304caaf1f680c12a69b10eb3760
Jul 28 20:47:50.802858 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:50.802831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"dd6ab2202a2073879aee143e1665e1371aa16304caaf1f680c12a69b10eb3760"}
Jul 28 20:47:52.462174 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.462124    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"]
Jul 28 20:47:52.464531 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.464508    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.467754 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.467732    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 20:47:52.468250 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.468231    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 20:47:52.468346 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.468288    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 20:47:52.468346 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.468298    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 20:47:52.468346 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.468329    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 20:47:52.468605 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.468589    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-478mc\""
Jul 28 20:47:52.481081 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.480889    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 20:47:52.488476 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.488453    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"]
Jul 28 20:47:52.498015 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.497996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.498115 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.498036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-federate-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.498115 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.498070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-serving-certs-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.498229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.498120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-metrics-client-ca\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.502153 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.498187    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.502153 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.499297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.502153 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.499357    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g8z6\" (UniqueName: \"kubernetes.io/projected/7603053a-c958-4c92-b291-aa103a1c62dd-kube-api-access-7g8z6\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.502153 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.499438    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600378 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-metrics-client-ca\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8z6\" (UniqueName: \"kubernetes.io/projected/7603053a-c958-4c92-b291-aa103a1c62dd-kube-api-access-7g8z6\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600517    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-federate-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.600574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.600542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-serving-certs-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.601209 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.601182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-metrics-client-ca\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.601539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.601303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-serving-certs-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.601654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.601574    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.603263 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.603239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.603722 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.603695    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-telemeter-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.603852 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.603831    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.603896 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.603837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7603053a-c958-4c92-b291-aa103a1c62dd-federate-client-tls\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.610153 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.610120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8z6\" (UniqueName: \"kubernetes.io/projected/7603053a-c958-4c92-b291-aa103a1c62dd-kube-api-access-7g8z6\") pod \"telemeter-client-6c5c69fbdd-dv8k8\" (UID: \"7603053a-c958-4c92-b291-aa103a1c62dd\") " pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.776121 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.776084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"
Jul 28 20:47:52.811758 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.811554    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cc04b02-32c0-4e69-b872-bb97fc3b58cf" containerID="dfa3a21afecd677136f5837dea898933487f0cebadf12936057dc5ea38814cd8" exitCode=0
Jul 28 20:47:52.811758 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.811644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerDied","Data":"dfa3a21afecd677136f5837dea898933487f0cebadf12936057dc5ea38814cd8"}
Jul 28 20:47:52.931966 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:52.931355    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8"]
Jul 28 20:47:53.818661 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:53.818620    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8" event={"ID":"7603053a-c958-4c92-b291-aa103a1c62dd","Type":"ContainerStarted","Data":"f3c6fb0984ba154b6da405501407d1498545b5bd1895e81b8afdcc75eb8f08ca"}
Jul 28 20:47:54.824924 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:54.824898    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"ffc9e3e26234a6338c4ece7cc061cf55b4bf28e2acd52c6ffd9e372e402e0410"}
Jul 28 20:47:54.826460 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:54.826440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8" event={"ID":"7603053a-c958-4c92-b291-aa103a1c62dd","Type":"ContainerStarted","Data":"40d36158efa05a019ba90f45d646a608b387cf2a4ecbd2f997c6c239f28ea786"}
Jul 28 20:47:55.832441 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.832402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8" event={"ID":"7603053a-c958-4c92-b291-aa103a1c62dd","Type":"ContainerStarted","Data":"729523794ae43b58c38f5b99ac16c9c860628f3199c6717e8b7bababa43c1a70"}
Jul 28 20:47:55.832860 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.832515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8" event={"ID":"7603053a-c958-4c92-b291-aa103a1c62dd","Type":"ContainerStarted","Data":"d5a274cdd1785db0ec280a2b7674b10bc28b2cd2a4538e19db9a25672e3b01aa"}
Jul 28 20:47:55.835662 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.835640    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"9884d143ecd4a16fedbce61d1fb04615bbbe5fef0f515d73cb4094e148b2fc19"}
Jul 28 20:47:55.835763 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.835697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"7770ff65b3edc1d95543e5d3d97ac76f53927e7e8a1523e52abfefcd41730b51"}
Jul 28 20:47:55.835763 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.835725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"fe0298cd57e84e524f92ba3514747c781921d8678ffdf27e75b50e89797d1ec6"}
Jul 28 20:47:55.835763 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.835748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"78232aadb1a89d1359292f3e44fec43a81846e197fd5aa2ccc9cc2c25b86e315"}
Jul 28 20:47:55.835858 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.835769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc04b02-32c0-4e69-b872-bb97fc3b58cf","Type":"ContainerStarted","Data":"81dd34d4233dd778afc5c09b5ca18cfa55d31c884f9155a384e7d22b8d139cc3"}
Jul 28 20:47:55.855691 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.855639    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c5c69fbdd-dv8k8" podStartSLOduration=2.059068618 podStartE2EDuration="3.855626289s" totalImagesPullingTime="1.796557671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:47:55.854180669 +0000 UTC m=+170.253237334" watchObservedRunningTime="2026-07-28 20:47:55.855626289 +0000 UTC m=+170.254682954"
Jul 28 20:47:55.887014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:55.886975    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.902348951 podStartE2EDuration="5.886962185s" totalImagesPullingTime="2.968454044s" totalInitContainerRuntime="16.15919ms" isStatefulPod=false podCreationTimestamp="2026-07-28 20:47:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:47:55.884860133 +0000 UTC m=+170.283916799" watchObservedRunningTime="2026-07-28 20:47:55.886962185 +0000 UTC m=+170.286018897"
Jul 28 20:47:56.689091 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.689060    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:47:56.691439 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.691413    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.705991 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.705968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:47:56.735847 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.735815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.735985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.735863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.735985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.735886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.735985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.735901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.735985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.735947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.736161 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.736006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg8w5\" (UniqueName: \"kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.736161 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.736060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.836543 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg8w5\" (UniqueName: \"kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.836543 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837089 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.836687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837515 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.837473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.837975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.837944    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.838647 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.838359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.838647 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.838504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.840514 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.840487    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.841122 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.841104    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:56.845643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:56.845622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg8w5\" (UniqueName: \"kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5\") pod \"console-6c6866cc46-ltskw\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") " pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:57.002652 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:57.002567    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:47:57.132131 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:57.132098    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:47:57.135901 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:47:57.135872    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c21c3db_c095_4386_9991_f80ca3e07abb.slice/crio-4de27878b7b20febfc2d79a9e272417dfa00295d7bf488ba28b94daf961555d1 WatchSource:0}: Error finding container 4de27878b7b20febfc2d79a9e272417dfa00295d7bf488ba28b94daf961555d1: Status 404 returned error can't find the container with id 4de27878b7b20febfc2d79a9e272417dfa00295d7bf488ba28b94daf961555d1
Jul 28 20:47:57.847805 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:57.847772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6866cc46-ltskw" event={"ID":"5c21c3db-c095-4386-9991-f80ca3e07abb","Type":"ContainerStarted","Data":"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"}
Jul 28 20:47:57.848254 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:57.847816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6866cc46-ltskw" event={"ID":"5c21c3db-c095-4386-9991-f80ca3e07abb","Type":"ContainerStarted","Data":"4de27878b7b20febfc2d79a9e272417dfa00295d7bf488ba28b94daf961555d1"}
Jul 28 20:47:57.870474 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:47:57.870429    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6866cc46-ltskw" podStartSLOduration=1.870415137 podStartE2EDuration="1.870415137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:47:57.869278558 +0000 UTC m=+172.268335222" watchObservedRunningTime="2026-07-28 20:47:57.870415137 +0000 UTC m=+172.269471801"
Jul 28 20:48:07.002827 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:07.002787    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:48:07.003321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:07.002879    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:48:07.007633 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:07.007614    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:48:07.891003 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:07.890979    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:48:07.959924 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:07.959894    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:48:32.980353 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:32.980282    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9fdcff5db-nnphp" podUID="0a34582e-fb12-45f6-b6b1-4601327e65e0" containerName="console" containerID="cri-o://1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f" gracePeriod=15
Jul 28 20:48:33.220941 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.220919    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:48:33.250330 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250253    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250330 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250340    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250383    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmxl2\" (UniqueName: \"kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250406    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250436    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250461    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config\") pod \"0a34582e-fb12-45f6-b6b1-4601327e65e0\" (UID: \"0a34582e-fb12-45f6-b6b1-4601327e65e0\") "
Jul 28 20:48:33.250753 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250692    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:48:33.250809 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250773    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:48:33.250809 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250781    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:48:33.250913 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.250803    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config" (OuterVolumeSpecName: "console-config") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:48:33.252696 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.252664    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:48:33.252838 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.252817    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2" (OuterVolumeSpecName: "kube-api-access-hmxl2") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "kube-api-access-hmxl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:48:33.252889 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.252842    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a34582e-fb12-45f6-b6b1-4601327e65e0" (UID: "0a34582e-fb12-45f6-b6b1-4601327e65e0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:48:33.351352 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351322    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-oauth-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351352 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351347    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-trusted-ca-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351352 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351357    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351367    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmxl2\" (UniqueName: \"kubernetes.io/projected/0a34582e-fb12-45f6-b6b1-4601327e65e0-kube-api-access-hmxl2\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351376    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351385    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.351586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.351392    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a34582e-fb12-45f6-b6b1-4601327e65e0-console-oauth-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:48:33.986193 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.986155    2567 generic.go:356] "Generic (PLEG): container finished" podID="0a34582e-fb12-45f6-b6b1-4601327e65e0" containerID="1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f" exitCode=0
Jul 28 20:48:33.986579 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.986202    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fdcff5db-nnphp"
Jul 28 20:48:33.986579 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.986237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdcff5db-nnphp" event={"ID":"0a34582e-fb12-45f6-b6b1-4601327e65e0","Type":"ContainerDied","Data":"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"}
Jul 28 20:48:33.986579 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.986289    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fdcff5db-nnphp" event={"ID":"0a34582e-fb12-45f6-b6b1-4601327e65e0","Type":"ContainerDied","Data":"7ebed4d8b6e8d51aaccb3fdb5e89f1103bddddddb24b916863989427e6f91e33"}
Jul 28 20:48:33.986579 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.986308    2567 scope.go:117] "RemoveContainer" containerID="1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"
Jul 28 20:48:33.995427 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.995411    2567 scope.go:117] "RemoveContainer" containerID="1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"
Jul 28 20:48:33.995681 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:48:33.995664    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f\": container with ID starting with 1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f not found: ID does not exist" containerID="1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"
Jul 28 20:48:33.995728 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:33.995689    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f"} err="failed to get container status \"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f\": rpc error: code = NotFound desc = could not find container \"1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f\": container with ID starting with 1258c3ac4c51cb4f9a10d428dea07a79182ab2ddf71caac758fe7727906f3a4f not found: ID does not exist"
Jul 28 20:48:34.010274 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:34.010252    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:48:34.015562 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:34.015543    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9fdcff5db-nnphp"]
Jul 28 20:48:34.078285 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:48:34.075775    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a34582e-fb12-45f6-b6b1-4601327e65e0" path="/var/lib/kubelet/pods/0a34582e-fb12-45f6-b6b1-4601327e65e0/volumes"
Jul 28 20:49:06.693778 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.693745    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:49:06.694180 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.694057    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a34582e-fb12-45f6-b6b1-4601327e65e0" containerName="console"
Jul 28 20:49:06.694180 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.694068    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a34582e-fb12-45f6-b6b1-4601327e65e0" containerName="console"
Jul 28 20:49:06.694180 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.694112    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a34582e-fb12-45f6-b6b1-4601327e65e0" containerName="console"
Jul 28 20:49:06.698123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.698105    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.729282 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.729259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:49:06.810744 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.810894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810770    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.810894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb4dj\" (UniqueName: \"kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.810894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.810894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810871    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.810894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.811065 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.810903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912198 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb4dj\" (UniqueName: \"kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.912355 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.912312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.913118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.913094    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.913118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.913107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.913282 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.913262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.913463 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.913443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.914965 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.914945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.915017 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.914951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:06.919725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:06.919700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb4dj\" (UniqueName: \"kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj\") pod \"console-744b9dc87b-8f2z2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") " pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:07.007496 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:07.007423    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:07.136780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:07.136635    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:49:07.139490 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:49:07.139465    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc536dea2_a606_49b5_a5b9_29483555d4b2.slice/crio-d579ef37c8651fda39ad7a5798f720821ce9904a4703ccb90c3e639dbf677467 WatchSource:0}: Error finding container d579ef37c8651fda39ad7a5798f720821ce9904a4703ccb90c3e639dbf677467: Status 404 returned error can't find the container with id d579ef37c8651fda39ad7a5798f720821ce9904a4703ccb90c3e639dbf677467
Jul 28 20:49:08.112791 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:08.112755    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b9dc87b-8f2z2" event={"ID":"c536dea2-a606-49b5-a5b9-29483555d4b2","Type":"ContainerStarted","Data":"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"}
Jul 28 20:49:08.113183 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:08.112811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b9dc87b-8f2z2" event={"ID":"c536dea2-a606-49b5-a5b9-29483555d4b2","Type":"ContainerStarted","Data":"d579ef37c8651fda39ad7a5798f720821ce9904a4703ccb90c3e639dbf677467"}
Jul 28 20:49:08.132227 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:08.132167    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744b9dc87b-8f2z2" podStartSLOduration=2.132127737 podStartE2EDuration="2.132127737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:49:08.131985662 +0000 UTC m=+242.531042328" watchObservedRunningTime="2026-07-28 20:49:08.132127737 +0000 UTC m=+242.531184403"
Jul 28 20:49:17.008097 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:17.008060    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:17.008601 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:17.008114    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:17.012438 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:17.012417    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:17.148611 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:17.148576    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:49:17.229237 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:17.229201    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:49:42.247797 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.247716    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c6866cc46-ltskw" podUID="5c21c3db-c095-4386-9991-f80ca3e07abb" containerName="console" containerID="cri-o://abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de" gracePeriod=15
Jul 28 20:49:42.492979 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.492955    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:49:42.586654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586616    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.586654 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586659    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.586856 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586818    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg8w5\" (UniqueName: \"kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.586856 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586850    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.586952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586879    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.586952 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586916    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:49:42.587058 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.586990    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.587058 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587028    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle\") pod \"5c21c3db-c095-4386-9991-f80ca3e07abb\" (UID: \"5c21c3db-c095-4386-9991-f80ca3e07abb\") "
Jul 28 20:49:42.587395 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587369    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca" (OuterVolumeSpecName: "service-ca") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587403    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config" (OuterVolumeSpecName: "console-config") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587527    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587583    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-oauth-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587596    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587604    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-console-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.587688 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.587612    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c21c3db-c095-4386-9991-f80ca3e07abb-trusted-ca-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.588859 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.588824    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:49:42.588964 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.588908    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5" (OuterVolumeSpecName: "kube-api-access-jg8w5") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "kube-api-access-jg8w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:49:42.589124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.589095    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5c21c3db-c095-4386-9991-f80ca3e07abb" (UID: "5c21c3db-c095-4386-9991-f80ca3e07abb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:49:42.688339 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.688305    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-oauth-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.688339 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.688334    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg8w5\" (UniqueName: \"kubernetes.io/projected/5c21c3db-c095-4386-9991-f80ca3e07abb-kube-api-access-jg8w5\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:42.688339 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:42.688344    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c21c3db-c095-4386-9991-f80ca3e07abb-console-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:49:43.242984 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.242953    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c21c3db-c095-4386-9991-f80ca3e07abb" containerID="abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de" exitCode=0
Jul 28 20:49:43.243189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.242996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6866cc46-ltskw" event={"ID":"5c21c3db-c095-4386-9991-f80ca3e07abb","Type":"ContainerDied","Data":"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"}
Jul 28 20:49:43.243189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.243004    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6866cc46-ltskw"
Jul 28 20:49:43.243189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.243033    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6866cc46-ltskw" event={"ID":"5c21c3db-c095-4386-9991-f80ca3e07abb","Type":"ContainerDied","Data":"4de27878b7b20febfc2d79a9e272417dfa00295d7bf488ba28b94daf961555d1"}
Jul 28 20:49:43.243189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.243053    2567 scope.go:117] "RemoveContainer" containerID="abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"
Jul 28 20:49:43.253830 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.253463    2567 scope.go:117] "RemoveContainer" containerID="abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"
Jul 28 20:49:43.254071 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:49:43.253803    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de\": container with ID starting with abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de not found: ID does not exist" containerID="abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"
Jul 28 20:49:43.254071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.253983    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de"} err="failed to get container status \"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de\": rpc error: code = NotFound desc = could not find container \"abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de\": container with ID starting with abbbee3a7fa770cbb85f46d133b959d150fe8de07bed6457ef95a9d2352ca9de not found: ID does not exist"
Jul 28 20:49:43.268908 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.268882    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:49:43.270615 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:43.270594    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c6866cc46-ltskw"]
Jul 28 20:49:44.074919 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:49:44.074885    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c21c3db-c095-4386-9991-f80ca3e07abb" path="/var/lib/kubelet/pods/5c21c3db-c095-4386-9991-f80ca3e07abb/volumes"
Jul 28 20:50:05.983415 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:05.983391    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 20:50:19.425128 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.425090    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"]
Jul 28 20:50:19.427370 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.425428    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c21c3db-c095-4386-9991-f80ca3e07abb" containerName="console"
Jul 28 20:50:19.427370 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.425441    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c21c3db-c095-4386-9991-f80ca3e07abb" containerName="console"
Jul 28 20:50:19.427370 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.425485    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c21c3db-c095-4386-9991-f80ca3e07abb" containerName="console"
Jul 28 20:50:19.428316 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.428301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.431126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.431103    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sb4zn\""
Jul 28 20:50:19.431267 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.431129    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:50:19.431727 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.431711    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:50:19.437714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.437688    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"]
Jul 28 20:50:19.596806 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.596765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.596980 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.596850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.596980 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.596882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnl4l\" (UniqueName: \"kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.697342 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.697265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.697342 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.697305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnl4l\" (UniqueName: \"kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.697507 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.697438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.697692 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.697675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.697755 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.697740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.706190 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.706163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnl4l\" (UniqueName: \"kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.738156 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.738122    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:19.875422 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.875388    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"]
Jul 28 20:50:19.878556 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:50:19.878526    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b96c577_bb0e_41e5_bb82_bcebae09b28c.slice/crio-9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45 WatchSource:0}: Error finding container 9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45: Status 404 returned error can't find the container with id 9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45
Jul 28 20:50:19.880475 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:19.880461    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:50:20.384419 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:20.384388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerStarted","Data":"9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45"}
Jul 28 20:50:27.415336 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:27.415298    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerID="4300715e7516b4df47ba07c02606558175305c38d25ab78512cb3231df3f2a85" exitCode=0
Jul 28 20:50:27.415838 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:27.415376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerDied","Data":"4300715e7516b4df47ba07c02606558175305c38d25ab78512cb3231df3f2a85"}
Jul 28 20:50:34.445973 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:34.445932    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerID="5b94401acfa650c0a25b036677a187b72a25c8ac61084569fcba5c7f12fa273a" exitCode=0
Jul 28 20:50:34.446380 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:34.446001    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerDied","Data":"5b94401acfa650c0a25b036677a187b72a25c8ac61084569fcba5c7f12fa273a"}
Jul 28 20:50:41.477593 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:41.477561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerStarted","Data":"432406e7cdf99871047b42fbe65b5206a5ae082c62271ca123ad1488a857e657"}
Jul 28 20:50:41.497643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:41.497596    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" podStartSLOduration=2.8413728750000002 podStartE2EDuration="22.497579721s" totalImagesPullingTime="19.656206846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:50:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:50:41.495993724 +0000 UTC m=+335.895050389" watchObservedRunningTime="2026-07-28 20:50:41.497579721 +0000 UTC m=+335.896636386"
Jul 28 20:50:42.483249 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:42.483218    2567 generic.go:356] "Generic (PLEG): container finished" podID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerID="432406e7cdf99871047b42fbe65b5206a5ae082c62271ca123ad1488a857e657" exitCode=0
Jul 28 20:50:42.483641 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:42.483304    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerDied","Data":"432406e7cdf99871047b42fbe65b5206a5ae082c62271ca123ad1488a857e657"}
Jul 28 20:50:43.613615 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.613596    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:43.696186 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.696159    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle\") pod \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") "
Jul 28 20:50:43.696310 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.696222    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnl4l\" (UniqueName: \"kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l\") pod \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") "
Jul 28 20:50:43.696310 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.696257    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util\") pod \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\" (UID: \"8b96c577-bb0e-41e5-bb82-bcebae09b28c\") "
Jul 28 20:50:43.696935 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.696855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle" (OuterVolumeSpecName: "bundle") pod "8b96c577-bb0e-41e5-bb82-bcebae09b28c" (UID: "8b96c577-bb0e-41e5-bb82-bcebae09b28c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:50:43.698444 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.698425    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l" (OuterVolumeSpecName: "kube-api-access-hnl4l") pod "8b96c577-bb0e-41e5-bb82-bcebae09b28c" (UID: "8b96c577-bb0e-41e5-bb82-bcebae09b28c"). InnerVolumeSpecName "kube-api-access-hnl4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:50:43.700349 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.700328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util" (OuterVolumeSpecName: "util") pod "8b96c577-bb0e-41e5-bb82-bcebae09b28c" (UID: "8b96c577-bb0e-41e5-bb82-bcebae09b28c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:50:43.797010 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.796982    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:50:43.797010 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.797010    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnl4l\" (UniqueName: \"kubernetes.io/projected/8b96c577-bb0e-41e5-bb82-bcebae09b28c-kube-api-access-hnl4l\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:50:43.797226 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:43.797020    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b96c577-bb0e-41e5-bb82-bcebae09b28c-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:50:44.492831 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:44.492750    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh" event={"ID":"8b96c577-bb0e-41e5-bb82-bcebae09b28c","Type":"ContainerDied","Data":"9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45"}
Jul 28 20:50:44.492831 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:44.492801    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psnvh"
Jul 28 20:50:44.492999 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:44.492837    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bca7422f1baf2f6d2ec44dd4718c8e468555af0f120017b07a280c5bdef8e45"
Jul 28 20:50:47.613672 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.613635    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"]
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614064    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="util"
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614081    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="util"
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614103    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="extract"
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614112    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="extract"
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614124    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="pull"
Jul 28 20:50:47.614124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614152    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="pull"
Jul 28 20:50:47.614442 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.614224    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b96c577-bb0e-41e5-bb82-bcebae09b28c" containerName="extract"
Jul 28 20:50:47.616330 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.616308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.618354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.618316    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5d28r\""
Jul 28 20:50:47.618470 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.618396    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 20:50:47.618545 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.618490    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:50:47.631517 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.631483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"]
Jul 28 20:50:47.725858 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.725823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/750dd914-7411-4931-ab28-3fb0e303ddf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.725999 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.725877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ntm\" (UniqueName: \"kubernetes.io/projected/750dd914-7411-4931-ab28-3fb0e303ddf4-kube-api-access-w9ntm\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.827296 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.827253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/750dd914-7411-4931-ab28-3fb0e303ddf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.827458 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.827314    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ntm\" (UniqueName: \"kubernetes.io/projected/750dd914-7411-4931-ab28-3fb0e303ddf4-kube-api-access-w9ntm\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.827695 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.827677    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/750dd914-7411-4931-ab28-3fb0e303ddf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.838516 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.838494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ntm\" (UniqueName: \"kubernetes.io/projected/750dd914-7411-4931-ab28-3fb0e303ddf4-kube-api-access-w9ntm\") pod \"cert-manager-operator-controller-manager-55c444bf86-kk2x6\" (UID: \"750dd914-7411-4931-ab28-3fb0e303ddf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:47.924787 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:47.924708    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"
Jul 28 20:50:48.058832 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:48.058809    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6"]
Jul 28 20:50:48.061964 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:50:48.061938    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750dd914_7411_4931_ab28_3fb0e303ddf4.slice/crio-5936139d02a8e39018306fdba1a503817131840d599866d9b8546dc4bb072d72 WatchSource:0}: Error finding container 5936139d02a8e39018306fdba1a503817131840d599866d9b8546dc4bb072d72: Status 404 returned error can't find the container with id 5936139d02a8e39018306fdba1a503817131840d599866d9b8546dc4bb072d72
Jul 28 20:50:48.508836 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:48.508799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6" event={"ID":"750dd914-7411-4931-ab28-3fb0e303ddf4","Type":"ContainerStarted","Data":"5936139d02a8e39018306fdba1a503817131840d599866d9b8546dc4bb072d72"}
Jul 28 20:50:53.531602 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:53.531552    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6" event={"ID":"750dd914-7411-4931-ab28-3fb0e303ddf4","Type":"ContainerStarted","Data":"c927eb5f852c0956722a67aa9872a29af196b44aa8c87b7cac08b89cc83f5ff8"}
Jul 28 20:50:53.555714 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:53.555663    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kk2x6" podStartSLOduration=1.721728444 podStartE2EDuration="6.55564907s" totalImagesPullingTime="4.833920626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:50:53.554250221 +0000 UTC m=+347.953306886" watchObservedRunningTime="2026-07-28 20:50:53.55564907 +0000 UTC m=+347.954705769"
Jul 28 20:50:54.734935 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.734896    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2q7hg"]
Jul 28 20:50:54.737022 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.737003    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:54.739392 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.739371    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-85cvt\""
Jul 28 20:50:54.739487 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.739430    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 20:50:54.739487 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.739459    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 20:50:54.746494 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.746473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2q7hg"]
Jul 28 20:50:54.887951 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.887917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:54.888119 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.887986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4slzc\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-kube-api-access-4slzc\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:54.989478 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.989389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4slzc\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-kube-api-access-4slzc\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:54.989478 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:54.989429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:55.005390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.005363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4slzc\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-kube-api-access-4slzc\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:55.015024 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.015003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02aef39e-626d-43d5-928f-95d3756bada9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2q7hg\" (UID: \"02aef39e-626d-43d5-928f-95d3756bada9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:55.056233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.056212    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:55.198374 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.198349    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2q7hg"]
Jul 28 20:50:55.201814 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:50:55.201787    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02aef39e_626d_43d5_928f_95d3756bada9.slice/crio-1399b6ffad93f25d44909e341c2466aef1d49705c6706c5acc5a750dd3550d9d WatchSource:0}: Error finding container 1399b6ffad93f25d44909e341c2466aef1d49705c6706c5acc5a750dd3550d9d: Status 404 returned error can't find the container with id 1399b6ffad93f25d44909e341c2466aef1d49705c6706c5acc5a750dd3550d9d
Jul 28 20:50:55.544181 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.544149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg" event={"ID":"02aef39e-626d-43d5-928f-95d3756bada9","Type":"ContainerStarted","Data":"1399b6ffad93f25d44909e341c2466aef1d49705c6706c5acc5a750dd3550d9d"}
Jul 28 20:50:55.824867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.824782    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"]
Jul 28 20:50:55.827805 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.827782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.829928 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.829874    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:50:55.829928 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.829906    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:50:55.830120 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.829874    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sb4zn\""
Jul 28 20:50:55.839291 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.839026    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"]
Jul 28 20:50:55.897859 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.897825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blbcs\" (UniqueName: \"kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.898011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.897890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.898011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.897938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.999084 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.999046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.999278 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.999181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blbcs\" (UniqueName: \"kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.999278 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.999238    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.999531 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.999508    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:55.999623 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:55.999606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:56.008188 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.008164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blbcs\" (UniqueName: \"kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:56.134773 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.134684    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"]
Jul 28 20:50:56.137777 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.137757    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.140390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.140331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:50:56.145592 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.145572    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"]
Jul 28 20:50:56.200861 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.200800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdjxb\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-kube-api-access-mdjxb\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.201029 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.200971    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.300724 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.300618    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"]
Jul 28 20:50:56.301418 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.301392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.301724 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.301459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdjxb\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-kube-api-access-mdjxb\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.302988 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:50:56.302959    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a23359_f94f_4ed0_90b1_ffcbb18adb26.slice/crio-dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383 WatchSource:0}: Error finding container dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383: Status 404 returned error can't find the container with id dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383
Jul 28 20:50:56.310500 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.310475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdjxb\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-kube-api-access-mdjxb\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.310725 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.310709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08c164ab-c5da-4f20-86da-73dde27eb2c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rwmwp\" (UID: \"08c164ab-c5da-4f20-86da-73dde27eb2c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.468331 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.468302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"
Jul 28 20:50:56.551573 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.551538    2567 generic.go:356] "Generic (PLEG): container finished" podID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerID="09ef60ed0c6fb792fedecb9997b40521c3e4157a436bc45c7a4a10f21d4798ce" exitCode=0
Jul 28 20:50:56.551750 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.551581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k" event={"ID":"40a23359-f94f-4ed0-90b1-ffcbb18adb26","Type":"ContainerDied","Data":"09ef60ed0c6fb792fedecb9997b40521c3e4157a436bc45c7a4a10f21d4798ce"}
Jul 28 20:50:56.551750 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.551640    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k" event={"ID":"40a23359-f94f-4ed0-90b1-ffcbb18adb26","Type":"ContainerStarted","Data":"dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383"}
Jul 28 20:50:56.614298 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:56.614276    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp"]
Jul 28 20:50:56.617363 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:50:56.617335    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08c164ab_c5da_4f20_86da_73dde27eb2c7.slice/crio-5cb387408bb79e43926793b440aa0d418ff9a1ee6037ec35cc425a8a4bab9ee2 WatchSource:0}: Error finding container 5cb387408bb79e43926793b440aa0d418ff9a1ee6037ec35cc425a8a4bab9ee2: Status 404 returned error can't find the container with id 5cb387408bb79e43926793b440aa0d418ff9a1ee6037ec35cc425a8a4bab9ee2
Jul 28 20:50:57.558008 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:57.557932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp" event={"ID":"08c164ab-c5da-4f20-86da-73dde27eb2c7","Type":"ContainerStarted","Data":"5cb387408bb79e43926793b440aa0d418ff9a1ee6037ec35cc425a8a4bab9ee2"}
Jul 28 20:50:58.569180 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:58.569120    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp" event={"ID":"08c164ab-c5da-4f20-86da-73dde27eb2c7","Type":"ContainerStarted","Data":"0051017c2da3be7532f9f6fcca88d96054d75ba0a108332c4a28cb85e551000f"}
Jul 28 20:50:58.570932 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:58.570904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg" event={"ID":"02aef39e-626d-43d5-928f-95d3756bada9","Type":"ContainerStarted","Data":"2090a63af4307ba285bbeb04559f28fd8f72134977104629472362257d8802de"}
Jul 28 20:50:58.571071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:58.571042    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:50:58.585679 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:58.585636    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rwmwp" podStartSLOduration=1.409803591 podStartE2EDuration="2.585620391s" totalImagesPullingTime="1.1758168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:50:58.58430072 +0000 UTC m=+352.983357386" watchObservedRunningTime="2026-07-28 20:50:58.585620391 +0000 UTC m=+352.984677055"
Jul 28 20:50:58.601545 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:58.601496    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg" podStartSLOduration=2.010404754 podStartE2EDuration="4.601484407s" totalImagesPullingTime="2.591079653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:50:58.600273682 +0000 UTC m=+352.999330347" watchObservedRunningTime="2026-07-28 20:50:58.601484407 +0000 UTC m=+353.000541073"
Jul 28 20:50:59.577031 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:59.577000    2567 generic.go:356] "Generic (PLEG): container finished" podID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerID="3f0a3444fa5c9a4e7da2fdf45c39a35f7a2014c252124182587bc0845ad986fc" exitCode=0
Jul 28 20:50:59.577509 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:50:59.577087    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k" event={"ID":"40a23359-f94f-4ed0-90b1-ffcbb18adb26","Type":"ContainerDied","Data":"3f0a3444fa5c9a4e7da2fdf45c39a35f7a2014c252124182587bc0845ad986fc"}
Jul 28 20:51:00.583371 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:00.583338    2567 generic.go:356] "Generic (PLEG): container finished" podID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerID="cb765977f5ed38b3aa0a1dc29cf0e13de5a3067d9e9342bbd6670a619c16c566" exitCode=0
Jul 28 20:51:00.583800 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:00.583423    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k" event={"ID":"40a23359-f94f-4ed0-90b1-ffcbb18adb26","Type":"ContainerDied","Data":"cb765977f5ed38b3aa0a1dc29cf0e13de5a3067d9e9342bbd6670a619c16c566"}
Jul 28 20:51:01.723643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.723619    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:51:01.850773 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.850679    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util\") pod \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") "
Jul 28 20:51:01.850773 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.850727    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle\") pod \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") "
Jul 28 20:51:01.851009 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.850786    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blbcs\" (UniqueName: \"kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs\") pod \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\" (UID: \"40a23359-f94f-4ed0-90b1-ffcbb18adb26\") "
Jul 28 20:51:01.851201 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.851181    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle" (OuterVolumeSpecName: "bundle") pod "40a23359-f94f-4ed0-90b1-ffcbb18adb26" (UID: "40a23359-f94f-4ed0-90b1-ffcbb18adb26"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:01.853021 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.852998    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs" (OuterVolumeSpecName: "kube-api-access-blbcs") pod "40a23359-f94f-4ed0-90b1-ffcbb18adb26" (UID: "40a23359-f94f-4ed0-90b1-ffcbb18adb26"). InnerVolumeSpecName "kube-api-access-blbcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:51:01.855223 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.855191    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util" (OuterVolumeSpecName: "util") pod "40a23359-f94f-4ed0-90b1-ffcbb18adb26" (UID: "40a23359-f94f-4ed0-90b1-ffcbb18adb26"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:01.952026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.951989    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:01.952026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.952017    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a23359-f94f-4ed0-90b1-ffcbb18adb26-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:01.952026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:01.952029    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blbcs\" (UniqueName: \"kubernetes.io/projected/40a23359-f94f-4ed0-90b1-ffcbb18adb26-kube-api-access-blbcs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:02.594052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:02.594022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k" event={"ID":"40a23359-f94f-4ed0-90b1-ffcbb18adb26","Type":"ContainerDied","Data":"dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383"}
Jul 28 20:51:02.594052 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:02.594055    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f64d7k"
Jul 28 20:51:02.594296 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:02.594070    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedf489a1bfe64e526cd03e44d207b4f7dc6266112bd53267648167c9e023383"
Jul 28 20:51:04.036678 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.036642    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4qp4l"]
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.036985    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="extract"
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.036998    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="extract"
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.037009    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="util"
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.037014    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="util"
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.037029    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="pull"
Jul 28 20:51:04.037041 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.037035    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="pull"
Jul 28 20:51:04.037251 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.037084    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a23359-f94f-4ed0-90b1-ffcbb18adb26" containerName="extract"
Jul 28 20:51:04.040798 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.040781    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.043625 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.043607    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2qgbq\""
Jul 28 20:51:04.051690 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.051669    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4qp4l"]
Jul 28 20:51:04.172165 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.172101    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.172165 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.172163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljrx7\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-kube-api-access-ljrx7\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.273239 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.273211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.273412 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.273245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljrx7\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-kube-api-access-ljrx7\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.280835 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.280809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljrx7\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-kube-api-access-ljrx7\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.280981 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.280962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8310890-5f6d-40a5-b398-c895050447ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-4qp4l\" (UID: \"b8310890-5f6d-40a5-b398-c895050447ec\") " pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.350023 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.349952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4qp4l"
Jul 28 20:51:04.483983 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.483957    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4qp4l"]
Jul 28 20:51:04.486214 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:51:04.486184    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8310890_5f6d_40a5_b398_c895050447ec.slice/crio-ff155e868061af58ce63d1333edf2a4048dc0a303061a6e34b87fadb2efb3ffd WatchSource:0}: Error finding container ff155e868061af58ce63d1333edf2a4048dc0a303061a6e34b87fadb2efb3ffd: Status 404 returned error can't find the container with id ff155e868061af58ce63d1333edf2a4048dc0a303061a6e34b87fadb2efb3ffd
Jul 28 20:51:04.580121 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.580096    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2q7hg"
Jul 28 20:51:04.608922 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.608840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4qp4l" event={"ID":"b8310890-5f6d-40a5-b398-c895050447ec","Type":"ContainerStarted","Data":"8667e6266c4c019838d61e6f578dc936f3e3c8ded8349e6452131dddd3f7fc85"}
Jul 28 20:51:04.608922 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.608886    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4qp4l" event={"ID":"b8310890-5f6d-40a5-b398-c895050447ec","Type":"ContainerStarted","Data":"ff155e868061af58ce63d1333edf2a4048dc0a303061a6e34b87fadb2efb3ffd"}
Jul 28 20:51:04.626094 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:04.626042    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4qp4l" podStartSLOduration=0.626026131 podStartE2EDuration="626.026131ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:51:04.624899833 +0000 UTC m=+359.023956498" watchObservedRunningTime="2026-07-28 20:51:04.626026131 +0000 UTC m=+359.025082797"
Jul 28 20:51:14.514274 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.514240    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"]
Jul 28 20:51:14.524321 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.524283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.525990 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.525954    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"]
Jul 28 20:51:14.526464 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.526440    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:51:14.526984 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.526965    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:51:14.527110 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.526971    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sb4zn\""
Jul 28 20:51:14.558931 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.558901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpnpj\" (UniqueName: \"kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.559067 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.558944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.559067 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.558962    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.659619 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.659571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpnpj\" (UniqueName: \"kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.659796 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.659635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.659796 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.659653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.659990 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.659976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.660028 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.660007    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.668014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.667987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpnpj\" (UniqueName: \"kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.835644 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.835556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:14.992640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:14.992604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"]
Jul 28 20:51:14.995783 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:51:14.995751    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9bc44ae_a0b7_42dc_8ca7_90a91caed652.slice/crio-a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1 WatchSource:0}: Error finding container a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1: Status 404 returned error can't find the container with id a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1
Jul 28 20:51:15.657928 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:15.657894    2567 generic.go:356] "Generic (PLEG): container finished" podID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerID="86a69d7f6c553553684ffd262a7d6cd2382b723c2fdb3377e1008c7aba058474" exitCode=0
Jul 28 20:51:15.658426 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:15.657960    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt" event={"ID":"b9bc44ae-a0b7-42dc-8ca7-90a91caed652","Type":"ContainerDied","Data":"86a69d7f6c553553684ffd262a7d6cd2382b723c2fdb3377e1008c7aba058474"}
Jul 28 20:51:15.658426 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:15.658024    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt" event={"ID":"b9bc44ae-a0b7-42dc-8ca7-90a91caed652","Type":"ContainerStarted","Data":"a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1"}
Jul 28 20:51:16.664698 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:16.664614    2567 generic.go:356] "Generic (PLEG): container finished" podID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerID="7df07ecb08d157c4b1ea9a47c7d5725cacec92e3bf24e8c6b8e2a43f4de75762" exitCode=0
Jul 28 20:51:16.665029 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:16.664707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt" event={"ID":"b9bc44ae-a0b7-42dc-8ca7-90a91caed652","Type":"ContainerDied","Data":"7df07ecb08d157c4b1ea9a47c7d5725cacec92e3bf24e8c6b8e2a43f4de75762"}
Jul 28 20:51:17.671606 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:17.671568    2567 generic.go:356] "Generic (PLEG): container finished" podID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerID="0a1321a1ddc037d815390bcdf894930f6e74ab4d6d5955b171087f935220b0f7" exitCode=0
Jul 28 20:51:17.672000 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:17.671645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt" event={"ID":"b9bc44ae-a0b7-42dc-8ca7-90a91caed652","Type":"ContainerDied","Data":"0a1321a1ddc037d815390bcdf894930f6e74ab4d6d5955b171087f935220b0f7"}
Jul 28 20:51:18.805360 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.805338    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:18.894653 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.894619    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpnpj\" (UniqueName: \"kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj\") pod \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") "
Jul 28 20:51:18.894841 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.894677    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle\") pod \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") "
Jul 28 20:51:18.894841 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.894705    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util\") pod \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\" (UID: \"b9bc44ae-a0b7-42dc-8ca7-90a91caed652\") "
Jul 28 20:51:18.895446 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.895353    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle" (OuterVolumeSpecName: "bundle") pod "b9bc44ae-a0b7-42dc-8ca7-90a91caed652" (UID: "b9bc44ae-a0b7-42dc-8ca7-90a91caed652"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:18.896863 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.896833    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj" (OuterVolumeSpecName: "kube-api-access-dpnpj") pod "b9bc44ae-a0b7-42dc-8ca7-90a91caed652" (UID: "b9bc44ae-a0b7-42dc-8ca7-90a91caed652"). InnerVolumeSpecName "kube-api-access-dpnpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:51:18.900016 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.899982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util" (OuterVolumeSpecName: "util") pod "b9bc44ae-a0b7-42dc-8ca7-90a91caed652" (UID: "b9bc44ae-a0b7-42dc-8ca7-90a91caed652"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:18.996304 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.996216    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:18.996304 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.996245    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:18.996304 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:18.996260    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpnpj\" (UniqueName: \"kubernetes.io/projected/b9bc44ae-a0b7-42dc-8ca7-90a91caed652-kube-api-access-dpnpj\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:51:19.682092 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:19.682061    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt"
Jul 28 20:51:19.682270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:19.682059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxzrdt" event={"ID":"b9bc44ae-a0b7-42dc-8ca7-90a91caed652","Type":"ContainerDied","Data":"a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1"}
Jul 28 20:51:19.682270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:19.682186    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a63593fdf1ff09cadfa6835a38622eb432d9b3e40c5fa69480520133d6b04ed1"
Jul 28 20:51:31.281818 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.281776    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"]
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282240    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="extract"
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282259    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="extract"
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282278    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="util"
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282286    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="util"
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282301    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="pull"
Jul 28 20:51:31.282320 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282309    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="pull"
Jul 28 20:51:31.282614 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.282400    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9bc44ae-a0b7-42dc-8ca7-90a91caed652" containerName="extract"
Jul 28 20:51:31.284277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.284257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.287014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.286991    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 20:51:31.287795 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.287778    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 20:51:31.287867 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.287836    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:51:31.287917 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.287838    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 20:51:31.288114 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.288095    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tkb25\""
Jul 28 20:51:31.289518 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.289500    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 20:51:31.298905 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.298886    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"]
Jul 28 20:51:31.399155 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.395525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.399155 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.395612    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-metrics-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.399155 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.395661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cf0c7ca-a908-40df-ac19-e44cf765175b-manager-config\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.399155 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.395695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6rvc\" (UniqueName: \"kubernetes.io/projected/6cf0c7ca-a908-40df-ac19-e44cf765175b-kube-api-access-k6rvc\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.496538 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.496503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.496708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.496559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-metrics-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.496708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.496599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cf0c7ca-a908-40df-ac19-e44cf765175b-manager-config\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.496708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.496628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6rvc\" (UniqueName: \"kubernetes.io/projected/6cf0c7ca-a908-40df-ac19-e44cf765175b-kube-api-access-k6rvc\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.497596 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.497570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cf0c7ca-a908-40df-ac19-e44cf765175b-manager-config\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.499289 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.499259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-metrics-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.499474 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.499443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cf0c7ca-a908-40df-ac19-e44cf765175b-cert\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.510158 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.510109    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6rvc\" (UniqueName: \"kubernetes.io/projected/6cf0c7ca-a908-40df-ac19-e44cf765175b-kube-api-access-k6rvc\") pod \"lws-controller-manager-57d8fcb7f5-czwbc\" (UID: \"6cf0c7ca-a908-40df-ac19-e44cf765175b\") " pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.593501 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.593417    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:31.749798 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:51:31.749772    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf0c7ca_a908_40df_ac19_e44cf765175b.slice/crio-d880d733ade3f01d93b981fcc7512a04124692af1fe3d108ef2460e5befc5ec0 WatchSource:0}: Error finding container d880d733ade3f01d93b981fcc7512a04124692af1fe3d108ef2460e5befc5ec0: Status 404 returned error can't find the container with id d880d733ade3f01d93b981fcc7512a04124692af1fe3d108ef2460e5befc5ec0
Jul 28 20:51:31.761195 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:31.761160    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"]
Jul 28 20:51:32.068123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.068088    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"]
Jul 28 20:51:32.070909 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.070889    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.073982 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.073962    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 20:51:32.074118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.073991    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 20:51:32.074118 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.074002    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 20:51:32.080628 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.077548    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 20:51:32.080628 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.077731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gm89g\""
Jul 28 20:51:32.098998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.098971    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"]
Jul 28 20:51:32.101233 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.101203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-tmp\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.101343 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.101235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6wxk\" (UniqueName: \"kubernetes.io/projected/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-kube-api-access-k6wxk\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.101343 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.101261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-manifests\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.101465 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.101344    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.101465 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.101403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.202856 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.202815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.203036 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.202881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.203036 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.202936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-tmp\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.203036 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.202964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6wxk\" (UniqueName: \"kubernetes.io/projected/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-kube-api-access-k6wxk\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.203036 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.203000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-manifests\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.203490 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.203467    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-manifests\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.205381 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.205363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-tmp\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.205735 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.205715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.205772 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.205718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.212922 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.212902    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6wxk\" (UniqueName: \"kubernetes.io/projected/a0c5f35e-9e99-4e02-931c-d4a311aa76a4-kube-api-access-k6wxk\") pod \"opendatahub-operator-controller-manager-5966989555-pxtgx\" (UID: \"a0c5f35e-9e99-4e02-931c-d4a311aa76a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.386924 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.386844    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:32.534187 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.534057    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"]
Jul 28 20:51:32.536932 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:51:32.536906    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c5f35e_9e99_4e02_931c_d4a311aa76a4.slice/crio-6a613218ab29bef13c3836bf9357932a3d40db5056d7baf7beaee4d98770b9b8 WatchSource:0}: Error finding container 6a613218ab29bef13c3836bf9357932a3d40db5056d7baf7beaee4d98770b9b8: Status 404 returned error can't find the container with id 6a613218ab29bef13c3836bf9357932a3d40db5056d7baf7beaee4d98770b9b8
Jul 28 20:51:32.742191 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.742081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx" event={"ID":"a0c5f35e-9e99-4e02-931c-d4a311aa76a4","Type":"ContainerStarted","Data":"6a613218ab29bef13c3836bf9357932a3d40db5056d7baf7beaee4d98770b9b8"}
Jul 28 20:51:32.743517 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:32.743470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc" event={"ID":"6cf0c7ca-a908-40df-ac19-e44cf765175b","Type":"ContainerStarted","Data":"d880d733ade3f01d93b981fcc7512a04124692af1fe3d108ef2460e5befc5ec0"}
Jul 28 20:51:33.751778 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:33.751701    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc" event={"ID":"6cf0c7ca-a908-40df-ac19-e44cf765175b","Type":"ContainerStarted","Data":"8333fb22a649f87656507a70e57ab046f8a1a46f931c6c3ddc86ecc25860ecbe"}
Jul 28 20:51:33.752260 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:33.751954    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:33.772702 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:33.772645    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc" podStartSLOduration=1.068038879 podStartE2EDuration="2.772623755s" totalImagesPullingTime="1.704584876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:51:33.771010468 +0000 UTC m=+388.170067133" watchObservedRunningTime="2026-07-28 20:51:33.772623755 +0000 UTC m=+388.171680420"
Jul 28 20:51:35.765569 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:35.765534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx" event={"ID":"a0c5f35e-9e99-4e02-931c-d4a311aa76a4","Type":"ContainerStarted","Data":"e79c0dd22afd732dae054737f9554dd8241216239abbfd76f979f328d5d77ddd"}
Jul 28 20:51:36.771795 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:36.771758    2567 generic.go:356] "Generic (PLEG): container finished" podID="a0c5f35e-9e99-4e02-931c-d4a311aa76a4" containerID="e79c0dd22afd732dae054737f9554dd8241216239abbfd76f979f328d5d77ddd" exitCode=0
Jul 28 20:51:36.772173 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:36.771801    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx" event={"ID":"a0c5f35e-9e99-4e02-931c-d4a311aa76a4","Type":"ContainerDied","Data":"e79c0dd22afd732dae054737f9554dd8241216239abbfd76f979f328d5d77ddd"}
Jul 28 20:51:37.778649 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:37.778613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx" event={"ID":"a0c5f35e-9e99-4e02-931c-d4a311aa76a4","Type":"ContainerStarted","Data":"17bcf99de1b132210aabbd6bcd62e49053c72f8d3ac645c81d6f11b1ee8ec32d"}
Jul 28 20:51:37.779025 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:37.778687    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:51:37.806611 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:37.806555    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx" podStartSLOduration=1.421788568 podStartE2EDuration="5.806534877s" totalImagesPullingTime="3.300642421s" totalInitContainerRuntime="1.084103888s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:51:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:51:37.805550319 +0000 UTC m=+392.204606986" watchObservedRunningTime="2026-07-28 20:51:37.806534877 +0000 UTC m=+392.205591541"
Jul 28 20:51:44.761987 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:44.761956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57d8fcb7f5-czwbc"
Jul 28 20:51:48.785770 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:51:48.785736    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5966989555-pxtgx"
Jul 28 20:52:02.513956 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.513908    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"]
Jul 28 20:52:02.517176 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.517153    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.532157 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.532121    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 20:52:02.532267 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.532150    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wpjm9\""
Jul 28 20:52:02.535056 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.535041    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 20:52:02.535230 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.535218    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 20:52:02.535313 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.535299    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 20:52:02.556539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.556518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"]
Jul 28 20:52:02.666673 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.666643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56474\" (UniqueName: \"kubernetes.io/projected/ebd954c0-7142-47a1-81d9-736a4a6890a2-kube-api-access-56474\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.666673 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.666675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd954c0-7142-47a1-81d9-736a4a6890a2-tls-certs\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.666887 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.666780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd954c0-7142-47a1-81d9-736a4a6890a2-tmp\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.767558 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.767461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd954c0-7142-47a1-81d9-736a4a6890a2-tmp\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.767739 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.767558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56474\" (UniqueName: \"kubernetes.io/projected/ebd954c0-7142-47a1-81d9-736a4a6890a2-kube-api-access-56474\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.767739 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.767591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd954c0-7142-47a1-81d9-736a4a6890a2-tls-certs\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.769980 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.769951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd954c0-7142-47a1-81d9-736a4a6890a2-tmp\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.770197 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.770179    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd954c0-7142-47a1-81d9-736a4a6890a2-tls-certs\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.785469 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.785441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56474\" (UniqueName: \"kubernetes.io/projected/ebd954c0-7142-47a1-81d9-736a4a6890a2-kube-api-access-56474\") pod \"kube-auth-proxy-77d6c74c55-45hwl\" (UID: \"ebd954c0-7142-47a1-81d9-736a4a6890a2\") " pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.826121 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.826095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"
Jul 28 20:52:02.979595 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:02.979569    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl"]
Jul 28 20:52:02.982038 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:52:02.982007    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd954c0_7142_47a1_81d9_736a4a6890a2.slice/crio-6ff14716b4422dd68c6995d88a39dc97fb965f7678fa73c60a8ac2afdc611a9c WatchSource:0}: Error finding container 6ff14716b4422dd68c6995d88a39dc97fb965f7678fa73c60a8ac2afdc611a9c: Status 404 returned error can't find the container with id 6ff14716b4422dd68c6995d88a39dc97fb965f7678fa73c60a8ac2afdc611a9c
Jul 28 20:52:03.916522 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:03.916486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl" event={"ID":"ebd954c0-7142-47a1-81d9-736a4a6890a2","Type":"ContainerStarted","Data":"6ff14716b4422dd68c6995d88a39dc97fb965f7678fa73c60a8ac2afdc611a9c"}
Jul 28 20:52:05.928296 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:05.928199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl" event={"ID":"ebd954c0-7142-47a1-81d9-736a4a6890a2","Type":"ContainerStarted","Data":"734336c5ef5393d020efc0481e587dd6376e7b869a27f71228913f3c5cb039b3"}
Jul 28 20:52:05.953020 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:52:05.952967    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77d6c74c55-45hwl" podStartSLOduration=1.285400499 podStartE2EDuration="3.952948417s" totalImagesPullingTime="2.667547918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:52:05.95217676 +0000 UTC m=+420.351233426" watchObservedRunningTime="2026-07-28 20:52:05.952948417 +0000 UTC m=+420.352005085"
Jul 28 20:53:08.429059 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.429028    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d4b79bd4-wlzbq"]
Jul 28 20:53:08.431313 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.431295    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.442536 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.442512    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4b79bd4-wlzbq"]
Jul 28 20:53:08.549539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62xc5\" (UniqueName: \"kubernetes.io/projected/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-kube-api-access-62xc5\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549540    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-oauth-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-oauth-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-trusted-ca-bundle\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-service-ca\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.549780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.549764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.629740 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.629709    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"]
Jul 28 20:53:08.632088 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.632073    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.634156 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.634115    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:53:08.634263 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.634113    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vzkpd\""
Jul 28 20:53:08.634263 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.634121    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:53:08.641300 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.641278    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"]
Jul 28 20:53:08.651064 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651044    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-oauth-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-trusted-ca-bundle\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-service-ca\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651303 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651303 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62xc5\" (UniqueName: \"kubernetes.io/projected/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-kube-api-access-62xc5\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651405 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-oauth-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651802 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-oauth-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651958 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651920    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-service-ca\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.651958 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.651931    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-trusted-ca-bundle\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.652198 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.652180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.654863 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.654830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-oauth-config\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.654966 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.654948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-console-serving-cert\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.659804 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.659779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62xc5\" (UniqueName: \"kubernetes.io/projected/5c1e1b9b-fa61-4f77-a19e-0e912e5101cd-kube-api-access-62xc5\") pod \"console-5d4b79bd4-wlzbq\" (UID: \"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd\") " pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.740973 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.740889    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:08.751859 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.751833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.751965 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.751870    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5n68\" (UniqueName: \"kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.751965 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.751947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.852436 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.852402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.852586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.852442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5n68\" (UniqueName: \"kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.852586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.852491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.852861 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.852847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.852898 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.852845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.860703 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.860676    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5n68\" (UniqueName: \"kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:08.871767 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.871737    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4b79bd4-wlzbq"]
Jul 28 20:53:08.875662 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:08.875633    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1e1b9b_fa61_4f77_a19e_0e912e5101cd.slice/crio-79a03de5a427966f786413f818655f4ec4db951d3747b8b5b4e5644266cda76e WatchSource:0}: Error finding container 79a03de5a427966f786413f818655f4ec4db951d3747b8b5b4e5644266cda76e: Status 404 returned error can't find the container with id 79a03de5a427966f786413f818655f4ec4db951d3747b8b5b4e5644266cda76e
Jul 28 20:53:08.942307 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:08.942282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:09.079305 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.079269    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"]
Jul 28 20:53:09.082367 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:09.082334    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38521bdd_0e98_416c_98f1_a8aaa3bfaf88.slice/crio-49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed WatchSource:0}: Error finding container 49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed: Status 404 returned error can't find the container with id 49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed
Jul 28 20:53:09.215161 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.215115    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"]
Jul 28 20:53:09.217960 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.217937    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b79bd4-wlzbq" event={"ID":"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd","Type":"ContainerStarted","Data":"4971c2d6f5944ea0a2ec620ace1ae2b2c5d29e7355aa832b12c895a37aaa8ee4"}
Jul 28 20:53:09.218087 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.217965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b79bd4-wlzbq" event={"ID":"5c1e1b9b-fa61-4f77-a19e-0e912e5101cd","Type":"ContainerStarted","Data":"79a03de5a427966f786413f818655f4ec4db951d3747b8b5b4e5644266cda76e"}
Jul 28 20:53:09.218087 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.218043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.218500 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.218483    2567 generic.go:356] "Generic (PLEG): container finished" podID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerID="388c117bc6f9418685063034e6950f0b5c20045ff5032a5887840162092c5612" exitCode=0
Jul 28 20:53:09.218577 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.218510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4" event={"ID":"38521bdd-0e98-416c-98f1-a8aaa3bfaf88","Type":"ContainerDied","Data":"388c117bc6f9418685063034e6950f0b5c20045ff5032a5887840162092c5612"}
Jul 28 20:53:09.218577 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.218533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4" event={"ID":"38521bdd-0e98-416c-98f1-a8aaa3bfaf88","Type":"ContainerStarted","Data":"49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed"}
Jul 28 20:53:09.226098 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.226077    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"]
Jul 28 20:53:09.232391 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.232343    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d4b79bd4-wlzbq" podStartSLOduration=1.2323266880000001 podStartE2EDuration="1.232326688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:09.231965815 +0000 UTC m=+483.631022483" watchObservedRunningTime="2026-07-28 20:53:09.232326688 +0000 UTC m=+483.631383354"
Jul 28 20:53:09.357791 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.357759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.357985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.357860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.357985 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.357894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2f9\" (UniqueName: \"kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.459043 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.459006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.459523 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.459066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.459523 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.459089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2f9\" (UniqueName: \"kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.459523 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.459457    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.459523 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.459471    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.467311 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.467281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2f9\" (UniqueName: \"kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.556974 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.556947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:09.689028 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.689004    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"]
Jul 28 20:53:09.690914 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:09.690883    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fbfc9be_b99f_46fc_8e7b_c8e0363c6fd1.slice/crio-ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae WatchSource:0}: Error finding container ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae: Status 404 returned error can't find the container with id ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae
Jul 28 20:53:09.817244 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.817175    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"]
Jul 28 20:53:09.819891 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.819870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:09.828353 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.828326    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"]
Jul 28 20:53:09.963298 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.963272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:09.963428 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.963351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:09.963511 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:09.963429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qccwr\" (UniqueName: \"kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.064582 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.064549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.064757 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.064613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qccwr\" (UniqueName: \"kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.064757 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.064654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.064986 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.064966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.065033 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.064988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.073298 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.073234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qccwr\" (UniqueName: \"kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.149256 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.149220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:10.224964 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.222969    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"]
Jul 28 20:53:10.232084 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.232003    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.233127 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.232593    2567 generic.go:356] "Generic (PLEG): container finished" podID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerID="38786bb0bd670f7446c50e011fe3aa0afa69c9b2827cc218f654e8b460b74caa" exitCode=0
Jul 28 20:53:10.233127 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.232691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d" event={"ID":"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1","Type":"ContainerDied","Data":"38786bb0bd670f7446c50e011fe3aa0afa69c9b2827cc218f654e8b460b74caa"}
Jul 28 20:53:10.233127 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.232732    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d" event={"ID":"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1","Type":"ContainerStarted","Data":"ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae"}
Jul 28 20:53:10.233411 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.233231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"]
Jul 28 20:53:10.236217 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.235306    2567 generic.go:356] "Generic (PLEG): container finished" podID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerID="6372f95ceed75b5d370c2ac1dac31fc3754a5f1d236ddde582357ba26efa2c50" exitCode=0
Jul 28 20:53:10.236424 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.236281    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4" event={"ID":"38521bdd-0e98-416c-98f1-a8aaa3bfaf88","Type":"ContainerDied","Data":"6372f95ceed75b5d370c2ac1dac31fc3754a5f1d236ddde582357ba26efa2c50"}
Jul 28 20:53:10.322852 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.322824    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"]
Jul 28 20:53:10.325641 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:10.325582    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1d9fff_25af_4d25_834e_1d89f24fc889.slice/crio-7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed WatchSource:0}: Error finding container 7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed: Status 404 returned error can't find the container with id 7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed
Jul 28 20:53:10.366845 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.366814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.366991 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.366907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.367070 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.367004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xnhx\" (UniqueName: \"kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.467659 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.467631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.467998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.467683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.467998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.467752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xnhx\" (UniqueName: \"kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.467998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.467950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.467998 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.467984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.476411 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.476393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xnhx\" (UniqueName: \"kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.544872 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.544841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:10.675880 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:10.675853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"]
Jul 28 20:53:10.679173 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:10.679150    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c9cff8_e09e_40e4_be58_70f725c0a8f7.slice/crio-00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124 WatchSource:0}: Error finding container 00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124: Status 404 returned error can't find the container with id 00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124
Jul 28 20:53:11.241291 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.241256    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerID="b9516fa7776f9c41462301cfa8bc0f21fd6cbe980b217eb35a53d2051dee8ee3" exitCode=0
Jul 28 20:53:11.241421 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.241311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerDied","Data":"b9516fa7776f9c41462301cfa8bc0f21fd6cbe980b217eb35a53d2051dee8ee3"}
Jul 28 20:53:11.241421 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.241361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerStarted","Data":"00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124"}
Jul 28 20:53:11.243410 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.243383    2567 generic.go:356] "Generic (PLEG): container finished" podID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerID="04ef8cb08572232a86a3be05b7f4adb4094af883e89eeec715c806dbfd592d32" exitCode=0
Jul 28 20:53:11.243534 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.243462    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d" event={"ID":"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1","Type":"ContainerDied","Data":"04ef8cb08572232a86a3be05b7f4adb4094af883e89eeec715c806dbfd592d32"}
Jul 28 20:53:11.245784 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.245764    2567 generic.go:356] "Generic (PLEG): container finished" podID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerID="2d4edd9542204cf0da4a991562afee7f149870e53827366cefbb1b8ce0d2901a" exitCode=0
Jul 28 20:53:11.245889 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.245828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4" event={"ID":"38521bdd-0e98-416c-98f1-a8aaa3bfaf88","Type":"ContainerDied","Data":"2d4edd9542204cf0da4a991562afee7f149870e53827366cefbb1b8ce0d2901a"}
Jul 28 20:53:11.247339 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.247322    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerID="a82fb6907c62f2ea2fe02af9d174468fa38b48acf2ec349a434ca227addf8784" exitCode=0
Jul 28 20:53:11.247419 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.247358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87" event={"ID":"0e1d9fff-25af-4d25-834e-1d89f24fc889","Type":"ContainerDied","Data":"a82fb6907c62f2ea2fe02af9d174468fa38b48acf2ec349a434ca227addf8784"}
Jul 28 20:53:11.247419 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:11.247381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87" event={"ID":"0e1d9fff-25af-4d25-834e-1d89f24fc889","Type":"ContainerStarted","Data":"7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed"}
Jul 28 20:53:12.254923 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.254895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerStarted","Data":"39b68f35754ed2dca82ea98075e0d5f0d74298f664cf88779862efcb02fd6c99"}
Jul 28 20:53:12.257301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.257217    2567 generic.go:356] "Generic (PLEG): container finished" podID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerID="935d54a4592e41340ea7ca21e75a541570d4abbf9b518cd6ea6ff35b9fdb2e11" exitCode=0
Jul 28 20:53:12.257301 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.257292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d" event={"ID":"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1","Type":"ContainerDied","Data":"935d54a4592e41340ea7ca21e75a541570d4abbf9b518cd6ea6ff35b9fdb2e11"}
Jul 28 20:53:12.422920 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.422859    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:12.485879 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.485852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5n68\" (UniqueName: \"kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68\") pod \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") "
Jul 28 20:53:12.485982 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.485895    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle\") pod \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") "
Jul 28 20:53:12.486040 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.485985    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util\") pod \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\" (UID: \"38521bdd-0e98-416c-98f1-a8aaa3bfaf88\") "
Jul 28 20:53:12.486529 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.486504    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle" (OuterVolumeSpecName: "bundle") pod "38521bdd-0e98-416c-98f1-a8aaa3bfaf88" (UID: "38521bdd-0e98-416c-98f1-a8aaa3bfaf88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:12.488494 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.488474    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68" (OuterVolumeSpecName: "kube-api-access-b5n68") pod "38521bdd-0e98-416c-98f1-a8aaa3bfaf88" (UID: "38521bdd-0e98-416c-98f1-a8aaa3bfaf88"). InnerVolumeSpecName "kube-api-access-b5n68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:12.492056 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.492036    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util" (OuterVolumeSpecName: "util") pod "38521bdd-0e98-416c-98f1-a8aaa3bfaf88" (UID: "38521bdd-0e98-416c-98f1-a8aaa3bfaf88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:12.587521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.587482    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5n68\" (UniqueName: \"kubernetes.io/projected/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-kube-api-access-b5n68\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:12.587521 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.587521    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:12.587686 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:12.587536    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38521bdd-0e98-416c-98f1-a8aaa3bfaf88-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:13.264586 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.264553    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerID="39b68f35754ed2dca82ea98075e0d5f0d74298f664cf88779862efcb02fd6c99" exitCode=0
Jul 28 20:53:13.264978 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.264649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerDied","Data":"39b68f35754ed2dca82ea98075e0d5f0d74298f664cf88779862efcb02fd6c99"}
Jul 28 20:53:13.266567 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.266542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4" event={"ID":"38521bdd-0e98-416c-98f1-a8aaa3bfaf88","Type":"ContainerDied","Data":"49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed"}
Jul 28 20:53:13.266567 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.266558    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4"
Jul 28 20:53:13.266712 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.266584    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c65416258d11f65032d8457c9da621d19ad2a6b36246b1e5f173db420834ed"
Jul 28 20:53:13.268571 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.268552    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerID="715e908a6c9c277e0689bd56d877c7007650289cd757d9f415de199d173f933e" exitCode=0
Jul 28 20:53:13.268642 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.268607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87" event={"ID":"0e1d9fff-25af-4d25-834e-1d89f24fc889","Type":"ContainerDied","Data":"715e908a6c9c277e0689bd56d877c7007650289cd757d9f415de199d173f933e"}
Jul 28 20:53:13.404673 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.404644    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:13.495499 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.495459    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util\") pod \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") "
Jul 28 20:53:13.495643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.495517    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn2f9\" (UniqueName: \"kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9\") pod \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") "
Jul 28 20:53:13.495643 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.495634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle\") pod \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\" (UID: \"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1\") "
Jul 28 20:53:13.496152 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.496100    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle" (OuterVolumeSpecName: "bundle") pod "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" (UID: "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:13.497668 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.497647    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9" (OuterVolumeSpecName: "kube-api-access-bn2f9") pod "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" (UID: "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1"). InnerVolumeSpecName "kube-api-access-bn2f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:13.500791 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.500771    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util" (OuterVolumeSpecName: "util") pod "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" (UID: "1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:13.596718 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.596639    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:13.596718 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.596665    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:13.596718 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:13.596675    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn2f9\" (UniqueName: \"kubernetes.io/projected/1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1-kube-api-access-bn2f9\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:14.275765 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.275736    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d"
Jul 28 20:53:14.275765 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.275739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d" event={"ID":"1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1","Type":"ContainerDied","Data":"ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae"}
Jul 28 20:53:14.276219 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.275787    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef14a871e820c13c0189df84f0bdf4c4f5a2d44b52c27bc7a99e3603d93b92ae"
Jul 28 20:53:14.277932 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.277895    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerID="c73279a9facc50f81abf102cb1391571ae48e9447b1df84f3aabc87a2eac30f6" exitCode=0
Jul 28 20:53:14.278054 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.278014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87" event={"ID":"0e1d9fff-25af-4d25-834e-1d89f24fc889","Type":"ContainerDied","Data":"c73279a9facc50f81abf102cb1391571ae48e9447b1df84f3aabc87a2eac30f6"}
Jul 28 20:53:14.280001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.279978    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerID="3c96c1a3e71446b56ee7c8a70ba5caf8d98647520050b0677a0f007b2ba84fa3" exitCode=0
Jul 28 20:53:14.280111 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:14.280036    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerDied","Data":"3c96c1a3e71446b56ee7c8a70ba5caf8d98647520050b0677a0f007b2ba84fa3"}
Jul 28 20:53:15.443169 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.443126    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:15.444179 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.444162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:15.513572 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513541    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle\") pod \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") "
Jul 28 20:53:15.513708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513606    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util\") pod \"0e1d9fff-25af-4d25-834e-1d89f24fc889\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") "
Jul 28 20:53:15.513708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qccwr\" (UniqueName: \"kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr\") pod \"0e1d9fff-25af-4d25-834e-1d89f24fc889\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") "
Jul 28 20:53:15.513708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513651    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util\") pod \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") "
Jul 28 20:53:15.513708 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513673    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xnhx\" (UniqueName: \"kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx\") pod \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\" (UID: \"83c9cff8-e09e-40e4-be58-70f725c0a8f7\") "
Jul 28 20:53:15.513930 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.513716    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle\") pod \"0e1d9fff-25af-4d25-834e-1d89f24fc889\" (UID: \"0e1d9fff-25af-4d25-834e-1d89f24fc889\") "
Jul 28 20:53:15.514124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.514097    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle" (OuterVolumeSpecName: "bundle") pod "83c9cff8-e09e-40e4-be58-70f725c0a8f7" (UID: "83c9cff8-e09e-40e4-be58-70f725c0a8f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:15.514244 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.514211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle" (OuterVolumeSpecName: "bundle") pod "0e1d9fff-25af-4d25-834e-1d89f24fc889" (UID: "0e1d9fff-25af-4d25-834e-1d89f24fc889"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:15.515969 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.515948    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx" (OuterVolumeSpecName: "kube-api-access-8xnhx") pod "83c9cff8-e09e-40e4-be58-70f725c0a8f7" (UID: "83c9cff8-e09e-40e4-be58-70f725c0a8f7"). InnerVolumeSpecName "kube-api-access-8xnhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:15.516062 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.516004    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr" (OuterVolumeSpecName: "kube-api-access-qccwr") pod "0e1d9fff-25af-4d25-834e-1d89f24fc889" (UID: "0e1d9fff-25af-4d25-834e-1d89f24fc889"). InnerVolumeSpecName "kube-api-access-qccwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:15.522114 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.522084    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util" (OuterVolumeSpecName: "util") pod "0e1d9fff-25af-4d25-834e-1d89f24fc889" (UID: "0e1d9fff-25af-4d25-834e-1d89f24fc889"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:15.522507 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.522482    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util" (OuterVolumeSpecName: "util") pod "83c9cff8-e09e-40e4-be58-70f725c0a8f7" (UID: "83c9cff8-e09e-40e4-be58-70f725c0a8f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614543    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xnhx\" (UniqueName: \"kubernetes.io/projected/83c9cff8-e09e-40e4-be58-70f725c0a8f7-kube-api-access-8xnhx\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614589    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614599    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614606    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d9fff-25af-4d25-834e-1d89f24fc889-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614614    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qccwr\" (UniqueName: \"kubernetes.io/projected/0e1d9fff-25af-4d25-834e-1d89f24fc889-kube-api-access-qccwr\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:15.614640 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:15.614622    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83c9cff8-e09e-40e4-be58-70f725c0a8f7-util\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:16.291807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.291777    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5"
Jul 28 20:53:16.291807 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.291791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5" event={"ID":"83c9cff8-e09e-40e4-be58-70f725c0a8f7","Type":"ContainerDied","Data":"00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124"}
Jul 28 20:53:16.292068 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.291846    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00839129345be85245551e3d49edcf2adb57ecdfaa72b7a97d50acaa36d39124"
Jul 28 20:53:16.293835 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.293811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87" event={"ID":"0e1d9fff-25af-4d25-834e-1d89f24fc889","Type":"ContainerDied","Data":"7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed"}
Jul 28 20:53:16.293835 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.293839    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87"
Jul 28 20:53:16.293835 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:16.293851    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e379aca79714ce936466ff44b220ba7b09e1ee3b8757cd34b660d0bf9f875ed"
Jul 28 20:53:18.741822 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:18.741782    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:18.741822 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:18.741839    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:18.746428 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:18.746409    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:19.312905 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:19.312880    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d4b79bd4-wlzbq"
Jul 28 20:53:19.359491 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:19.359459    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:53:36.527241 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527209    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"]
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527586    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527601    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527612    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527617    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527624    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="pull"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527630    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="pull"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527636    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="extract"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527641    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="extract"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527660    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527668    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="util"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527678    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="pull"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527684    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="pull"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527695    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="extract"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527703    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="extract"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527713    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="extract"
Jul 28 20:53:36.527721 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527721    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527731    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="pull"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527738    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="pull"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527750    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527757    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527763    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="util"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527767    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="util"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527773    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="pull"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527777    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="pull"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527841    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c9cff8-e09e-40e4-be58-70f725c0a8f7" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527850    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="38521bdd-0e98-416c-98f1-a8aaa3bfaf88" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527857    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1" containerName="extract"
Jul 28 20:53:36.528270 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.527863    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1d9fff-25af-4d25-834e-1d89f24fc889" containerName="extract"
Jul 28 20:53:36.530034 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.530012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:36.532460 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.532442    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wlfhb\""
Jul 28 20:53:36.532653 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.532640    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:53:36.533011 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.532997    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:53:36.546575 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.546556    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"]
Jul 28 20:53:36.596008 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.595984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdssm\" (UniqueName: \"kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm\") pod \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" (UID: \"6c756994-76b3-47c6-83cc-581d9455862d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:36.696588 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.696561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdssm\" (UniqueName: \"kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm\") pod \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" (UID: \"6c756994-76b3-47c6-83cc-581d9455862d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:36.704113 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.704089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdssm\" (UniqueName: \"kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm\") pod \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" (UID: \"6c756994-76b3-47c6-83cc-581d9455862d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:36.839792 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.839714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:36.970729 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:36.970701    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"]
Jul 28 20:53:36.974497 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:36.974465    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c756994_76b3_47c6_83cc_581d9455862d.slice/crio-660e1929d5690d34b91ff3665ce67df9d07465ef8efcbb1158c04bb6641ea7c8 WatchSource:0}: Error finding container 660e1929d5690d34b91ff3665ce67df9d07465ef8efcbb1158c04bb6641ea7c8: Status 404 returned error can't find the container with id 660e1929d5690d34b91ff3665ce67df9d07465ef8efcbb1158c04bb6641ea7c8
Jul 28 20:53:37.405055 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:37.405016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg" event={"ID":"6c756994-76b3-47c6-83cc-581d9455862d","Type":"ContainerStarted","Data":"660e1929d5690d34b91ff3665ce67df9d07465ef8efcbb1158c04bb6641ea7c8"}
Jul 28 20:53:39.417277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:39.417233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg" event={"ID":"6c756994-76b3-47c6-83cc-581d9455862d","Type":"ContainerStarted","Data":"53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e"}
Jul 28 20:53:39.417701 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:39.417377    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:39.433126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:39.433072    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg" podStartSLOduration=1.5035121280000001 podStartE2EDuration="3.433056469s" totalImagesPullingTime="1.929544341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:39.432287455 +0000 UTC m=+513.831344121" watchObservedRunningTime="2026-07-28 20:53:39.433056469 +0000 UTC m=+513.832113134"
Jul 28 20:53:40.739808 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.739778    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"]
Jul 28 20:53:40.742364 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.742344    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.744730 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.744707    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 20:53:40.744730 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.744722    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vzkpd\""
Jul 28 20:53:40.744893 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.744743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 20:53:40.751308 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.751287    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"]
Jul 28 20:53:40.828825 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.828790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.828969 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.828833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85j5\" (UniqueName: \"kubernetes.io/projected/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-kube-api-access-m85j5\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.828969 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.828939    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.930277 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.930244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.930437 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.930321    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.930437 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.930343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85j5\" (UniqueName: \"kubernetes.io/projected/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-kube-api-access-m85j5\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.930983 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.930915    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.932827 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.932802    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:40.937582 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:40.937564    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85j5\" (UniqueName: \"kubernetes.io/projected/a5d1c8c2-ff5e-4513-8b31-9c6da80d1399-kube-api-access-m85j5\") pod \"kuadrant-console-plugin-66b5774985-d46jc\" (UID: \"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:41.052121 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:41.052096    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"
Jul 28 20:53:41.181539 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:41.181514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc"]
Jul 28 20:53:41.183534 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:53:41.183507    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5d1c8c2_ff5e_4513_8b31_9c6da80d1399.slice/crio-fa61df3e79092f9b250a35809eaa15aecc17be388ed8717ebe302f47981318d3 WatchSource:0}: Error finding container fa61df3e79092f9b250a35809eaa15aecc17be388ed8717ebe302f47981318d3: Status 404 returned error can't find the container with id fa61df3e79092f9b250a35809eaa15aecc17be388ed8717ebe302f47981318d3
Jul 28 20:53:41.428738 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:41.428654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc" event={"ID":"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399","Type":"ContainerStarted","Data":"fa61df3e79092f9b250a35809eaa15aecc17be388ed8717ebe302f47981318d3"}
Jul 28 20:53:44.379464 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:44.379424    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-744b9dc87b-8f2z2" podUID="c536dea2-a606-49b5-a5b9-29483555d4b2" containerName="console" containerID="cri-o://295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f" gracePeriod=15
Jul 28 20:53:46.168520 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.168495    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:53:46.282164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282059    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282151    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282217    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282244    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282281    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282305    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb4dj\" (UniqueName: \"kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282367    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config\") pod \"c536dea2-a606-49b5-a5b9-29483555d4b2\" (UID: \"c536dea2-a606-49b5-a5b9-29483555d4b2\") "
Jul 28 20:53:46.282645 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282462    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:46.282784 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282759    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config" (OuterVolumeSpecName: "console-config") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:46.282880 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282849    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:46.282939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282924    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:46.282994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282928    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-oauth-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.282994 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.282969    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-console-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.285123 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.285097    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:53:46.285344 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.285318    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:53:46.285545 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.285523    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj" (OuterVolumeSpecName: "kube-api-access-cb4dj") pod "c536dea2-a606-49b5-a5b9-29483555d4b2" (UID: "c536dea2-a606-49b5-a5b9-29483555d4b2"). InnerVolumeSpecName "kube-api-access-cb4dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:46.384462 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.384427    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.384462 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.384454    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-serving-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.384462 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.384463    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c536dea2-a606-49b5-a5b9-29483555d4b2-console-oauth-config\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.384687 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.384475    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c536dea2-a606-49b5-a5b9-29483555d4b2-trusted-ca-bundle\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.384687 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.384489    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb4dj\" (UniqueName: \"kubernetes.io/projected/c536dea2-a606-49b5-a5b9-29483555d4b2-kube-api-access-cb4dj\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:53:46.459513 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.459465    2567 generic.go:356] "Generic (PLEG): container finished" podID="c536dea2-a606-49b5-a5b9-29483555d4b2" containerID="295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f" exitCode=0
Jul 28 20:53:46.459685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.459514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b9dc87b-8f2z2" event={"ID":"c536dea2-a606-49b5-a5b9-29483555d4b2","Type":"ContainerDied","Data":"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"}
Jul 28 20:53:46.459685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.459542    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b9dc87b-8f2z2"
Jul 28 20:53:46.459685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.459577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b9dc87b-8f2z2" event={"ID":"c536dea2-a606-49b5-a5b9-29483555d4b2","Type":"ContainerDied","Data":"d579ef37c8651fda39ad7a5798f720821ce9904a4703ccb90c3e639dbf677467"}
Jul 28 20:53:46.459685 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.459603    2567 scope.go:117] "RemoveContainer" containerID="295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"
Jul 28 20:53:46.471253 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.471203    2567 scope.go:117] "RemoveContainer" containerID="295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"
Jul 28 20:53:46.471514 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:53:46.471482    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f\": container with ID starting with 295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f not found: ID does not exist" containerID="295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"
Jul 28 20:53:46.471607 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.471520    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f"} err="failed to get container status \"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f\": rpc error: code = NotFound desc = could not find container \"295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f\": container with ID starting with 295c8857c58e0d311180ce2d097f483d68802ac75e48c9433f6faf357833706f not found: ID does not exist"
Jul 28 20:53:46.485214 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.485185    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:53:46.488748 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:46.488727    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744b9dc87b-8f2z2"]
Jul 28 20:53:48.075882 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:48.075849    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c536dea2-a606-49b5-a5b9-29483555d4b2" path="/var/lib/kubelet/pods/c536dea2-a606-49b5-a5b9-29483555d4b2/volumes"
Jul 28 20:53:50.425622 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:50.425591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:53:51.490344 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:51.490294    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"]
Jul 28 20:53:51.490796 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:51.490530    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg" podUID="6c756994-76b3-47c6-83cc-581d9455862d" containerName="manager" containerID="cri-o://53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e" gracePeriod=2
Jul 28 20:53:51.500957 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:53:51.500932    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"]
Jul 28 20:54:02.518502 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.518480    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:02.520636 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:02.520600    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="6c756994-76b3-47c6-83cc-581d9455862d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:02.556944 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.556922    2567 generic.go:356] "Generic (PLEG): container finished" podID="6c756994-76b3-47c6-83cc-581d9455862d" containerID="53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e" exitCode=0
Jul 28 20:54:02.557057 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.556957    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:02.557057 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.557024    2567 scope.go:117] "RemoveContainer" containerID="53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e"
Jul 28 20:54:02.558591 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.558570    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc" event={"ID":"a5d1c8c2-ff5e-4513-8b31-9c6da80d1399","Type":"ContainerStarted","Data":"6bf73e7a3ac4bf0730b39a02d1a880192bca9507aced837d9340332d0f488fcc"}
Jul 28 20:54:02.559177 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:02.559152    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="6c756994-76b3-47c6-83cc-581d9455862d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:02.566550 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.566532    2567 scope.go:117] "RemoveContainer" containerID="53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e"
Jul 28 20:54:02.566854 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:02.566832    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e\": container with ID starting with 53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e not found: ID does not exist" containerID="53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e"
Jul 28 20:54:02.566933 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.566864    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e"} err="failed to get container status \"53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e\": rpc error: code = NotFound desc = could not find container \"53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e\": container with ID starting with 53ea37dc7a8f97274fd15a975824d600509cf6dfa40e38fa244624d5abfdab6e not found: ID does not exist"
Jul 28 20:54:02.576283 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:02.576259    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="6c756994-76b3-47c6-83cc-581d9455862d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:02.577053 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.577016    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d46jc" podStartSLOduration=1.322605685 podStartE2EDuration="22.577004742s" totalImagesPullingTime="21.254399057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:02.57443346 +0000 UTC m=+536.973490120" watchObservedRunningTime="2026-07-28 20:54:02.577004742 +0000 UTC m=+536.976061432"
Jul 28 20:54:02.633762 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.633685    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdssm\" (UniqueName: \"kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm\") pod \"6c756994-76b3-47c6-83cc-581d9455862d\" (UID: \"6c756994-76b3-47c6-83cc-581d9455862d\") "
Jul 28 20:54:02.635780 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.635756    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm" (OuterVolumeSpecName: "kube-api-access-zdssm") pod "6c756994-76b3-47c6-83cc-581d9455862d" (UID: "6c756994-76b3-47c6-83cc-581d9455862d"). InnerVolumeSpecName "kube-api-access-zdssm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:02.735046 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:02.735018    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdssm\" (UniqueName: \"kubernetes.io/projected/6c756994-76b3-47c6-83cc-581d9455862d-kube-api-access-zdssm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:54:02.867692 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:02.867661    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qw7jg\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="6c756994-76b3-47c6-83cc-581d9455862d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qw7jg"
Jul 28 20:54:04.076124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:04.076091    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c756994-76b3-47c6-83cc-581d9455862d" path="/var/lib/kubelet/pods/6c756994-76b3-47c6-83cc-581d9455862d/volumes"
Jul 28 20:54:34.841469 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841435    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841806    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c756994-76b3-47c6-83cc-581d9455862d" containerName="manager"
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841818    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c756994-76b3-47c6-83cc-581d9455862d" containerName="manager"
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841828    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c536dea2-a606-49b5-a5b9-29483555d4b2" containerName="console"
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841838    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c536dea2-a606-49b5-a5b9-29483555d4b2" containerName="console"
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841895    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c756994-76b3-47c6-83cc-581d9455862d" containerName="manager"
Jul 28 20:54:34.841921 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.841906    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c536dea2-a606-49b5-a5b9-29483555d4b2" containerName="console"
Jul 28 20:54:34.886481 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.886451    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:54:34.886481 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.886484    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:54:34.886663 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.886567    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:34.888665 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.888641    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 20:54:34.901157 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.901115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38b0c4d7-bd28-486b-9960-3803a9d0e893-config-file\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:34.901247 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:34.901174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xjpt\" (UniqueName: \"kubernetes.io/projected/38b0c4d7-bd28-486b-9960-3803a9d0e893-kube-api-access-4xjpt\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.002070 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.002030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xjpt\" (UniqueName: \"kubernetes.io/projected/38b0c4d7-bd28-486b-9960-3803a9d0e893-kube-api-access-4xjpt\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.002264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.002116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38b0c4d7-bd28-486b-9960-3803a9d0e893-config-file\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.002701 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.002684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38b0c4d7-bd28-486b-9960-3803a9d0e893-config-file\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.010470 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.010441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xjpt\" (UniqueName: \"kubernetes.io/projected/38b0c4d7-bd28-486b-9960-3803a9d0e893-kube-api-access-4xjpt\") pod \"limitador-limitador-54d6d988d5-vfzvg\" (UID: \"38b0c4d7-bd28-486b-9960-3803a9d0e893\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.196399 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.196297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:35.344064 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.344039    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:54:35.345764 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:54:35.345740    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b0c4d7_bd28_486b_9960_3803a9d0e893.slice/crio-e83b87f3d8611f888e209ba75e8b934dff1475498253841515b3c6ea1f358b97 WatchSource:0}: Error finding container e83b87f3d8611f888e209ba75e8b934dff1475498253841515b3c6ea1f358b97: Status 404 returned error can't find the container with id e83b87f3d8611f888e209ba75e8b934dff1475498253841515b3c6ea1f358b97
Jul 28 20:54:35.346078 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.346059    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:35.355533 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.355514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:35.357707 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.357690    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wnbsb\""
Jul 28 20:54:35.362540 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.362514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:35.405943 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.405919    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mvwk\" (UniqueName: \"kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk\") pod \"authorino-58cd955675-2gdbp\" (UID: \"a246ac53-fa67-49ca-85b6-8e9fc428a54d\") " pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:35.507087 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.507005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mvwk\" (UniqueName: \"kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk\") pod \"authorino-58cd955675-2gdbp\" (UID: \"a246ac53-fa67-49ca-85b6-8e9fc428a54d\") " pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:35.515894 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.515867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mvwk\" (UniqueName: \"kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk\") pod \"authorino-58cd955675-2gdbp\" (UID: \"a246ac53-fa67-49ca-85b6-8e9fc428a54d\") " pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:35.665516 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.665483    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:35.732164 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.732062    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg" event={"ID":"38b0c4d7-bd28-486b-9960-3803a9d0e893","Type":"ContainerStarted","Data":"e83b87f3d8611f888e209ba75e8b934dff1475498253841515b3c6ea1f358b97"}
Jul 28 20:54:35.741513 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.741478    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:54:35.744515 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.744494    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:54:35.752787 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.752728    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:54:35.809413 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.809284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjq7j\" (UniqueName: \"kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j\") pod \"authorino-75d556bd77-69rt7\" (UID: \"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d\") " pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:54:35.822390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.822365    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:35.825128 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:54:35.825099    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda246ac53_fa67_49ca_85b6_8e9fc428a54d.slice/crio-10c1f9b2c2f078eb482ac0ee0f7c66ddb43fead6aa597076a9e9fda85286778e WatchSource:0}: Error finding container 10c1f9b2c2f078eb482ac0ee0f7c66ddb43fead6aa597076a9e9fda85286778e: Status 404 returned error can't find the container with id 10c1f9b2c2f078eb482ac0ee0f7c66ddb43fead6aa597076a9e9fda85286778e
Jul 28 20:54:35.910486 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.910447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjq7j\" (UniqueName: \"kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j\") pod \"authorino-75d556bd77-69rt7\" (UID: \"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d\") " pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:54:35.924017 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:35.923959    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjq7j\" (UniqueName: \"kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j\") pod \"authorino-75d556bd77-69rt7\" (UID: \"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d\") " pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:54:36.058939 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:36.058908    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:54:36.216055 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:36.216027    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:54:36.218040 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:54:36.218004    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda822d6f4_7b76_4d6e_8734_5aca2aa2ee3d.slice/crio-1d2887625b27b4bf3094cbe96b8cc2c94423ea9f138be9db804cba574c0e62e3 WatchSource:0}: Error finding container 1d2887625b27b4bf3094cbe96b8cc2c94423ea9f138be9db804cba574c0e62e3: Status 404 returned error can't find the container with id 1d2887625b27b4bf3094cbe96b8cc2c94423ea9f138be9db804cba574c0e62e3
Jul 28 20:54:36.742891 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:36.742850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2gdbp" event={"ID":"a246ac53-fa67-49ca-85b6-8e9fc428a54d","Type":"ContainerStarted","Data":"10c1f9b2c2f078eb482ac0ee0f7c66ddb43fead6aa597076a9e9fda85286778e"}
Jul 28 20:54:36.744617 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:36.744587    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-69rt7" event={"ID":"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d","Type":"ContainerStarted","Data":"1d2887625b27b4bf3094cbe96b8cc2c94423ea9f138be9db804cba574c0e62e3"}
Jul 28 20:54:40.771545 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.771491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-69rt7" event={"ID":"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d","Type":"ContainerStarted","Data":"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"}
Jul 28 20:54:40.773004 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.772974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2gdbp" event={"ID":"a246ac53-fa67-49ca-85b6-8e9fc428a54d","Type":"ContainerStarted","Data":"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"}
Jul 28 20:54:40.774354 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.774329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg" event={"ID":"38b0c4d7-bd28-486b-9960-3803a9d0e893","Type":"ContainerStarted","Data":"8e67d0038cb536ac2c30c7888fe8dd2f491f864aa6c930ca272e025c32004b89"}
Jul 28 20:54:40.774459 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.774447    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:54:40.786347 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.786303    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-69rt7" podStartSLOduration=2.296163737 podStartE2EDuration="5.786291121s" totalImagesPullingTime="3.490127384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:40.786070558 +0000 UTC m=+575.185127225" watchObservedRunningTime="2026-07-28 20:54:40.786291121 +0000 UTC m=+575.185347793"
Jul 28 20:54:40.801611 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.801571    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2gdbp" podStartSLOduration=1.9035402829999999 podStartE2EDuration="5.801558385s" totalImagesPullingTime="3.898018102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:40.799159466 +0000 UTC m=+575.198216131" watchObservedRunningTime="2026-07-28 20:54:40.801558385 +0000 UTC m=+575.200615027"
Jul 28 20:54:40.818023 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.817982    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg" podStartSLOduration=2.45621526 podStartE2EDuration="6.817968818s" totalImagesPullingTime="4.361753558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:40.815792614 +0000 UTC m=+575.214849280" watchObservedRunningTime="2026-07-28 20:54:40.817968818 +0000 UTC m=+575.217025482"
Jul 28 20:54:40.834084 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:40.834056    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:42.784186 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:42.784148    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2gdbp" podUID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" containerName="authorino" containerID="cri-o://163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d" gracePeriod=30
Jul 28 20:54:43.027241 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.027218    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:43.076513 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.076445    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mvwk\" (UniqueName: \"kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk\") pod \"a246ac53-fa67-49ca-85b6-8e9fc428a54d\" (UID: \"a246ac53-fa67-49ca-85b6-8e9fc428a54d\") "
Jul 28 20:54:43.078507 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.078484    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk" (OuterVolumeSpecName: "kube-api-access-2mvwk") pod "a246ac53-fa67-49ca-85b6-8e9fc428a54d" (UID: "a246ac53-fa67-49ca-85b6-8e9fc428a54d"). InnerVolumeSpecName "kube-api-access-2mvwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:43.177494 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.177470    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mvwk\" (UniqueName: \"kubernetes.io/projected/a246ac53-fa67-49ca-85b6-8e9fc428a54d-kube-api-access-2mvwk\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:54:43.789884 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.789852    2567 generic.go:356] "Generic (PLEG): container finished" podID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" containerID="163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d" exitCode=0
Jul 28 20:54:43.790286 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.789891    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2gdbp"
Jul 28 20:54:43.790286 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.789938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2gdbp" event={"ID":"a246ac53-fa67-49ca-85b6-8e9fc428a54d","Type":"ContainerDied","Data":"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"}
Jul 28 20:54:43.790286 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.789976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2gdbp" event={"ID":"a246ac53-fa67-49ca-85b6-8e9fc428a54d","Type":"ContainerDied","Data":"10c1f9b2c2f078eb482ac0ee0f7c66ddb43fead6aa597076a9e9fda85286778e"}
Jul 28 20:54:43.790286 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.789993    2567 scope.go:117] "RemoveContainer" containerID="163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"
Jul 28 20:54:43.798967 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.798953    2567 scope.go:117] "RemoveContainer" containerID="163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"
Jul 28 20:54:43.799185 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:54:43.799169    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d\": container with ID starting with 163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d not found: ID does not exist" containerID="163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"
Jul 28 20:54:43.799240 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.799192    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d"} err="failed to get container status \"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d\": rpc error: code = NotFound desc = could not find container \"163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d\": container with ID starting with 163b831db3dd6f7dec987a11388fc45fd850ee67c24fbd446e8c4cc74e1ac27d not found: ID does not exist"
Jul 28 20:54:43.809587 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.809566    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:43.811530 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:43.811509    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2gdbp"]
Jul 28 20:54:44.076864 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:44.076788    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" path="/var/lib/kubelet/pods/a246ac53-fa67-49ca-85b6-8e9fc428a54d/volumes"
Jul 28 20:54:51.780567 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:54:51.780539    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"
Jul 28 20:55:07.653713 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.653673    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8jpwr"]
Jul 28 20:55:07.654251 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.654191    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" containerName="authorino"
Jul 28 20:55:07.654251 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.654245    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" containerName="authorino"
Jul 28 20:55:07.654390 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.654367    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a246ac53-fa67-49ca-85b6-8e9fc428a54d" containerName="authorino"
Jul 28 20:55:07.664886 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.664852    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8jpwr"]
Jul 28 20:55:07.665028 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.664961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.799091 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.799055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cds64\" (UniqueName: \"kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64\") pod \"authorino-5d4b67c446-8jpwr\" (UID: \"c6d9e589-6f6c-47f8-964e-e2437894cdd2\") " pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.841292 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.841258    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8jpwr"]
Jul 28 20:55:07.841527 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:55:07.841508    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cds64], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-8jpwr" podUID="c6d9e589-6f6c-47f8-964e-e2437894cdd2"
Jul 28 20:55:07.867660 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.867632    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-q7xpf"]
Jul 28 20:55:07.870992 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.870972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:07.873212 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.873189    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 20:55:07.879346 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.879325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-q7xpf"]
Jul 28 20:55:07.899978 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.899958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cds64\" (UniqueName: \"kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64\") pod \"authorino-5d4b67c446-8jpwr\" (UID: \"c6d9e589-6f6c-47f8-964e-e2437894cdd2\") " pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.910903 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.910852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cds64\" (UniqueName: \"kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64\") pod \"authorino-5d4b67c446-8jpwr\" (UID: \"c6d9e589-6f6c-47f8-964e-e2437894cdd2\") " pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.912626 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.912609    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.923704 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.923686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:07.923910 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.923893    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-q7xpf"]
Jul 28 20:55:07.924159 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:55:07.924124    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7db6d tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-q7xpf" podUID="5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd"
Jul 28 20:55:07.951420 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.951395    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:55:07.954552 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.954530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:07.964353 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:07.964334    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:55:08.001304 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.001279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7db6d\" (UniqueName: \"kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.001420 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.001329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.102547 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.102523    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cds64\" (UniqueName: \"kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64\") pod \"c6d9e589-6f6c-47f8-964e-e2437894cdd2\" (UID: \"c6d9e589-6f6c-47f8-964e-e2437894cdd2\") "
Jul 28 20:55:08.102703 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.102654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7db6d\" (UniqueName: \"kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.102703 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.102697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.102814 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.102724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.102814 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.102778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwv6h\" (UniqueName: \"kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.104795 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.104765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64" (OuterVolumeSpecName: "kube-api-access-cds64") pod "c6d9e589-6f6c-47f8-964e-e2437894cdd2" (UID: "c6d9e589-6f6c-47f8-964e-e2437894cdd2"). InnerVolumeSpecName "kube-api-access-cds64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:08.105124 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.105109    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.111440 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.111420    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7db6d\" (UniqueName: \"kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d\") pod \"authorino-76877994-q7xpf\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") " pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.203569 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.203510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.203569 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.203548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwv6h\" (UniqueName: \"kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.203705 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.203622    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cds64\" (UniqueName: \"kubernetes.io/projected/c6d9e589-6f6c-47f8-964e-e2437894cdd2-kube-api-access-cds64\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:55:08.205763 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.205747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.211387 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.211365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwv6h\" (UniqueName: \"kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h\") pod \"authorino-7b78895d4c-5gfjw\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") " pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.264071 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.264047    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:55:08.398565 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.398537    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:55:08.400049 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:55:08.400018    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05b072d2_f21e_4d40_82b4_2739702a8c80.slice/crio-418ef49c20732bb76f6def67291c7ab72755f06203157336f048e8c23dd1d977 WatchSource:0}: Error finding container 418ef49c20732bb76f6def67291c7ab72755f06203157336f048e8c23dd1d977: Status 404 returned error can't find the container with id 418ef49c20732bb76f6def67291c7ab72755f06203157336f048e8c23dd1d977
Jul 28 20:55:08.919607 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.919572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" event={"ID":"05b072d2-f21e-4d40-82b4-2739702a8c80","Type":"ContainerStarted","Data":"418ef49c20732bb76f6def67291c7ab72755f06203157336f048e8c23dd1d977"}
Jul 28 20:55:08.920126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.919621    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8jpwr"
Jul 28 20:55:08.920126 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.919585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.929463 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.929446    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:08.960226 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.960202    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8jpwr"]
Jul 28 20:55:08.966749 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:08.966726    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8jpwr"]
Jul 28 20:55:09.110222 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.110191    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert\") pod \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") "
Jul 28 20:55:09.110380 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.110247    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7db6d\" (UniqueName: \"kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d\") pod \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\" (UID: \"5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd\") "
Jul 28 20:55:09.112349 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.112317    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d" (OuterVolumeSpecName: "kube-api-access-7db6d") pod "5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd" (UID: "5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd"). InnerVolumeSpecName "kube-api-access-7db6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:09.112349 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.112332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd" (UID: "5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:55:09.211274 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.211193    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-tls-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:55:09.211274 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.211222    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7db6d\" (UniqueName: \"kubernetes.io/projected/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd-kube-api-access-7db6d\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:55:09.925975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.925946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q7xpf"
Jul 28 20:55:09.925975 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.925946    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" event={"ID":"05b072d2-f21e-4d40-82b4-2739702a8c80","Type":"ContainerStarted","Data":"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"}
Jul 28 20:55:09.941959 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.941913    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" podStartSLOduration=2.387752066 podStartE2EDuration="2.941896988s" totalImagesPullingTime="554.144922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:55:09.940572786 +0000 UTC m=+604.339629452" watchObservedRunningTime="2026-07-28 20:55:09.941896988 +0000 UTC m=+604.340953653"
Jul 28 20:55:09.964829 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.964801    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:55:09.964996 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.964972    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-69rt7" podUID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" containerName="authorino" containerID="cri-o://6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659" gracePeriod=30
Jul 28 20:55:09.978014 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.977981    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-q7xpf"]
Jul 28 20:55:09.980489 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:09.980468    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-q7xpf"]
Jul 28 20:55:10.075875 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.075842    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd" path="/var/lib/kubelet/pods/5574aa38-5ac6-4c56-a9ef-0c5ca85da5dd/volumes"
Jul 28 20:55:10.076077 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.076065    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d9e589-6f6c-47f8-964e-e2437894cdd2" path="/var/lib/kubelet/pods/c6d9e589-6f6c-47f8-964e-e2437894cdd2/volumes"
Jul 28 20:55:10.284524 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.284498    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:55:10.424388 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.424357    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjq7j\" (UniqueName: \"kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j\") pod \"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d\" (UID: \"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d\") "
Jul 28 20:55:10.426470 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.426444    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j" (OuterVolumeSpecName: "kube-api-access-sjq7j") pod "a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" (UID: "a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d"). InnerVolumeSpecName "kube-api-access-sjq7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:10.525715 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.525692    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjq7j\" (UniqueName: \"kubernetes.io/projected/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d-kube-api-access-sjq7j\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:55:10.932001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.931918    2567 generic.go:356] "Generic (PLEG): container finished" podID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" containerID="6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659" exitCode=0
Jul 28 20:55:10.932001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.931962    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-69rt7"
Jul 28 20:55:10.932001 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.931998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-69rt7" event={"ID":"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d","Type":"ContainerDied","Data":"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"}
Jul 28 20:55:10.932493 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.932029    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-69rt7" event={"ID":"a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d","Type":"ContainerDied","Data":"1d2887625b27b4bf3094cbe96b8cc2c94423ea9f138be9db804cba574c0e62e3"}
Jul 28 20:55:10.932493 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.932047    2567 scope.go:117] "RemoveContainer" containerID="6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"
Jul 28 20:55:10.940715 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.940701    2567 scope.go:117] "RemoveContainer" containerID="6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"
Jul 28 20:55:10.940934 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:55:10.940918    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659\": container with ID starting with 6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659 not found: ID does not exist" containerID="6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"
Jul 28 20:55:10.940978 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.940941    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659"} err="failed to get container status \"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659\": rpc error: code = NotFound desc = could not find container \"6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659\": container with ID starting with 6f4603a6e2d934530740e742ea578d4fd802a7da2fab6fc41f1428ccbd205659 not found: ID does not exist"
Jul 28 20:55:10.952956 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.952932    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:55:10.956756 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:10.956737    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-69rt7"]
Jul 28 20:55:12.075941 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:12.075908    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" path="/var/lib/kubelet/pods/a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d/volumes"
Jul 28 20:55:52.095397 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:55:52.095364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:56:19.640531 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:19.640496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:56:34.541517 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:34.541482    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:56:44.227015 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.226982    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"]
Jul 28 20:56:44.227454 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.227427    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" containerName="authorino"
Jul 28 20:56:44.227454 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.227445    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" containerName="authorino"
Jul 28 20:56:44.227574 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.227559    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a822d6f4-7b76-4d6e-8734-5aca2aa2ee3d" containerName="authorino"
Jul 28 20:56:44.229769 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.229751    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.231770 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.231743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 20:56:44.232681 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.232658    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 20:56:44.232787 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.232689    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 20:56:44.232787 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.232694    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hkgj4\""
Jul 28 20:56:44.242597 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.242574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"]
Jul 28 20:56:44.274073 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.274229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.274229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821e1404-344d-4f62-a156-130a80590e31-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.274229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzp9n\" (UniqueName: \"kubernetes.io/projected/821e1404-344d-4f62-a156-130a80590e31-kube-api-access-hzp9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.274229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.274367 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.274240    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375202 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375417 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375231    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375417 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821e1404-344d-4f62-a156-130a80590e31-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375417 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzp9n\" (UniqueName: \"kubernetes.io/projected/821e1404-344d-4f62-a156-130a80590e31-kube-api-access-hzp9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375593 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375593 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375701 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375795 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.375858 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.375775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.377659 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.377635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821e1404-344d-4f62-a156-130a80590e31-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.377762 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.377742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821e1404-344d-4f62-a156-130a80590e31-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.385426 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.385404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzp9n\" (UniqueName: \"kubernetes.io/projected/821e1404-344d-4f62-a156-130a80590e31-kube-api-access-hzp9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7cj72\" (UID: \"821e1404-344d-4f62-a156-130a80590e31\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.542195 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.542163    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:44.671576 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.671550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"]
Jul 28 20:56:44.672854 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:56:44.672815    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821e1404_344d_4f62_a156_130a80590e31.slice/crio-d0c55cef2d7d85ef9d3b593be3426cb6ecf8baeccf3495ccde8b7a9ec31475d0 WatchSource:0}: Error finding container d0c55cef2d7d85ef9d3b593be3426cb6ecf8baeccf3495ccde8b7a9ec31475d0: Status 404 returned error can't find the container with id d0c55cef2d7d85ef9d3b593be3426cb6ecf8baeccf3495ccde8b7a9ec31475d0
Jul 28 20:56:44.674236 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:44.674220    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:56:45.433799 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:45.431559    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72" event={"ID":"821e1404-344d-4f62-a156-130a80590e31","Type":"ContainerStarted","Data":"d0c55cef2d7d85ef9d3b593be3426cb6ecf8baeccf3495ccde8b7a9ec31475d0"}
Jul 28 20:56:45.446152 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:45.446096    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:56:46.438380 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:46.438347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72" event={"ID":"821e1404-344d-4f62-a156-130a80590e31","Type":"ContainerStarted","Data":"863f9b8259ddbf24586d0e8fec44531869db63b14d9f0301924572736540dc0e"}
Jul 28 20:56:46.455866 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:46.455821    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72" podStartSLOduration=1.323005324 podStartE2EDuration="2.455806713s" totalImagesPullingTime="1.132801389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:56:46.453735947 +0000 UTC m=+700.852792612" watchObservedRunningTime="2026-07-28 20:56:46.455806713 +0000 UTC m=+700.854863377"
Jul 28 20:56:54.542932 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:54.542899    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:54.543424 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:54.542957    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:54.555731 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:54.555703    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:56:54.936403 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:54.936326    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:56:55.500231 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:56:55.500195    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7cj72"
Jul 28 20:57:05.551208 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:05.548211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:57:16.544346 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:16.544309    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:57:47.935757 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:47.935716    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fd47df9db-lvlfm"]
Jul 28 20:57:47.939219 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:47.939190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:47.945823 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:47.945797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd47df9db-lvlfm"]
Jul 28 20:57:48.027730 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.027696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd714256-4d62-4f1e-ac14-37d45943d479-tls-cert\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.027916 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.027754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn62l\" (UniqueName: \"kubernetes.io/projected/fd714256-4d62-4f1e-ac14-37d45943d479-kube-api-access-jn62l\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.128572 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.128539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd714256-4d62-4f1e-ac14-37d45943d479-tls-cert\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.128747 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.128594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn62l\" (UniqueName: \"kubernetes.io/projected/fd714256-4d62-4f1e-ac14-37d45943d479-kube-api-access-jn62l\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.131069 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.131048    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd714256-4d62-4f1e-ac14-37d45943d479-tls-cert\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.137026 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.137007    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn62l\" (UniqueName: \"kubernetes.io/projected/fd714256-4d62-4f1e-ac14-37d45943d479-kube-api-access-jn62l\") pod \"authorino-fd47df9db-lvlfm\" (UID: \"fd714256-4d62-4f1e-ac14-37d45943d479\") " pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.249416 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.249331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd47df9db-lvlfm"
Jul 28 20:57:48.398343 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.398314    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd47df9db-lvlfm"]
Jul 28 20:57:48.400360 ip-10-0-140-162 kubenswrapper[2567]: W0728 20:57:48.400335    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd714256_4d62_4f1e_ac14_37d45943d479.slice/crio-fa9c5de2c5ecb3e16f6451deb4758b1be68b10eb8d3634407420338f20072ce6 WatchSource:0}: Error finding container fa9c5de2c5ecb3e16f6451deb4758b1be68b10eb8d3634407420338f20072ce6: Status 404 returned error can't find the container with id fa9c5de2c5ecb3e16f6451deb4758b1be68b10eb8d3634407420338f20072ce6
Jul 28 20:57:48.772973 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:48.772934    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd47df9db-lvlfm" event={"ID":"fd714256-4d62-4f1e-ac14-37d45943d479","Type":"ContainerStarted","Data":"fa9c5de2c5ecb3e16f6451deb4758b1be68b10eb8d3634407420338f20072ce6"}
Jul 28 20:57:49.782522 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:49.782481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd47df9db-lvlfm" event={"ID":"fd714256-4d62-4f1e-ac14-37d45943d479","Type":"ContainerStarted","Data":"4be944356a7573b57e25ce8ff950483190e60f86b14c388c97cc76044873ffcd"}
Jul 28 20:57:49.817229 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:49.817151    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fd47df9db-lvlfm" podStartSLOduration=2.181339374 podStartE2EDuration="2.817113676s" totalImagesPullingTime="635.774302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:57:49.810497293 +0000 UTC m=+764.209553957" watchObservedRunningTime="2026-07-28 20:57:49.817113676 +0000 UTC m=+764.216170342"
Jul 28 20:57:49.879250 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:49.879219    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:57:49.879439 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:49.879399    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" podUID="05b072d2-f21e-4d40-82b4-2739702a8c80" containerName="authorino" containerID="cri-o://6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9" gracePeriod=30
Jul 28 20:57:50.135121 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.133958    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:57:50.248189 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.248157    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwv6h\" (UniqueName: \"kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h\") pod \"05b072d2-f21e-4d40-82b4-2739702a8c80\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") "
Jul 28 20:57:50.248375 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.248239    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert\") pod \"05b072d2-f21e-4d40-82b4-2739702a8c80\" (UID: \"05b072d2-f21e-4d40-82b4-2739702a8c80\") "
Jul 28 20:57:50.250407 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.250377    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h" (OuterVolumeSpecName: "kube-api-access-xwv6h") pod "05b072d2-f21e-4d40-82b4-2739702a8c80" (UID: "05b072d2-f21e-4d40-82b4-2739702a8c80"). InnerVolumeSpecName "kube-api-access-xwv6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:57:50.258515 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.258485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "05b072d2-f21e-4d40-82b4-2739702a8c80" (UID: "05b072d2-f21e-4d40-82b4-2739702a8c80"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:57:50.349416 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.349330    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwv6h\" (UniqueName: \"kubernetes.io/projected/05b072d2-f21e-4d40-82b4-2739702a8c80-kube-api-access-xwv6h\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:57:50.349416 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.349355    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05b072d2-f21e-4d40-82b4-2739702a8c80-tls-cert\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 20:57:50.789107 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.789069    2567 generic.go:356] "Generic (PLEG): container finished" podID="05b072d2-f21e-4d40-82b4-2739702a8c80" containerID="6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9" exitCode=0
Jul 28 20:57:50.789594 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.789117    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78895d4c-5gfjw"
Jul 28 20:57:50.789594 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.789162    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" event={"ID":"05b072d2-f21e-4d40-82b4-2739702a8c80","Type":"ContainerDied","Data":"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"}
Jul 28 20:57:50.789594 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.789210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78895d4c-5gfjw" event={"ID":"05b072d2-f21e-4d40-82b4-2739702a8c80","Type":"ContainerDied","Data":"418ef49c20732bb76f6def67291c7ab72755f06203157336f048e8c23dd1d977"}
Jul 28 20:57:50.789594 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.789233    2567 scope.go:117] "RemoveContainer" containerID="6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"
Jul 28 20:57:50.800496 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.800473    2567 scope.go:117] "RemoveContainer" containerID="6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"
Jul 28 20:57:50.800849 ip-10-0-140-162 kubenswrapper[2567]: E0728 20:57:50.800831    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9\": container with ID starting with 6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9 not found: ID does not exist" containerID="6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"
Jul 28 20:57:50.800906 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.800859    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9"} err="failed to get container status \"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9\": rpc error: code = NotFound desc = could not find container \"6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9\": container with ID starting with 6cc8082b52cc76c0b76746305b1de767e35bc3e58688580af297b176b49f24a9 not found: ID does not exist"
Jul 28 20:57:50.808891 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.808868    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:57:50.812444 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:50.812424    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b78895d4c-5gfjw"]
Jul 28 20:57:52.077264 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:57:52.077236    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05b072d2-f21e-4d40-82b4-2739702a8c80" path="/var/lib/kubelet/pods/05b072d2-f21e-4d40-82b4-2739702a8c80/volumes"
Jul 28 20:58:19.645680 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:58:19.645641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:58:29.041593 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:58:29.041511    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 20:58:38.944293 ip-10-0-140-162 kubenswrapper[2567]: I0728 20:58:38.944254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:00:18.845276 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:00:18.845237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:00:29.344865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:00:29.344830    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:04.247202 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:04.247167    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:14.239626 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:14.239592    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:22.844541 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:22.844507    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:31.936351 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:31.936278    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:41.845248 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:41.845210    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:01:52.455408 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:01:52.455370    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:02:03.543221 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:02:03.543181    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:02:13.543789 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:02:13.543755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:02:35.766367 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:02:35.766334    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:02:44.949640 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:02:44.949605    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:03:09.246252 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:03:09.246212    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:03:19.836981 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:03:19.836946    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:03:29.141184 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:03:29.141151    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:03:45.441911 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:03:45.441877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:03:56.539668 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:03:56.539632    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:04:06.149441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:04:06.149400    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:04:15.440194 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:04:15.440156    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:04:24.436319 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:04:24.436280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:06:14.504154 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:06:14.504104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:06:25.641584 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:06:25.641545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:06:40.553788 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:06:40.553750    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:06:50.243485 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:06:50.243440    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:01.642549 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:01.642515    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:10.743166 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:10.743119    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:21.635182 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:21.635151    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:32.744309 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:32.744232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:41.643693 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:41.643658    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:07:52.348043 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:07:52.348008    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:08:03.542258 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:03.542222    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:08:11.243802 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:11.243768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:08:29.841064 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:29.841028    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:08:39.643018 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:39.642983    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:08:44.136514 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.136482    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:08:44.136870 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.136812    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05b072d2-f21e-4d40-82b4-2739702a8c80" containerName="authorino"
Jul 28 21:08:44.136870 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.136822    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b072d2-f21e-4d40-82b4-2739702a8c80" containerName="authorino"
Jul 28 21:08:44.136945 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.136888    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="05b072d2-f21e-4d40-82b4-2739702a8c80" containerName="authorino"
Jul 28 21:08:44.139871 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.139852    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.141979 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.141957    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f4f83f49-kserve-self-signed-certs\""
Jul 28 21:08:44.160954 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.160932    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:08:44.217582 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.217722 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217596    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.217722 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.217722 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pdlq\" (UniqueName: \"kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.217822 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.217822 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.217772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318410 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318359    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318410 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318659 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pdlq\" (UniqueName: \"kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318659 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318659 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318659 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318869 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318869 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.318966 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.318905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.320867 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.320836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.321110 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.321095    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.326380 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.326361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pdlq\" (UniqueName: \"kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq\") pod \"e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") " pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.448878 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.448798    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:44.586547 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:44.586524    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:08:44.587954 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:08:44.587916    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4f68590_23f1_4cf9_97e7_e6f89ccf2df6.slice/crio-67b813ccd4d73d8f36e3ea41be353710b3f35733c4e57befbca82f58c77ee944 WatchSource:0}: Error finding container 67b813ccd4d73d8f36e3ea41be353710b3f35733c4e57befbca82f58c77ee944: Status 404 returned error can't find the container with id 67b813ccd4d73d8f36e3ea41be353710b3f35733c4e57befbca82f58c77ee944
Jul 28 21:08:45.177855 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:45.177822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" event={"ID":"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6","Type":"ContainerStarted","Data":"cea1be478d88329a39040629449f1e20434ffc00b8f7e2bdf4ccb7ea8d1fc00c"}
Jul 28 21:08:45.177855 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:45.177860    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" event={"ID":"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6","Type":"ContainerStarted","Data":"67b813ccd4d73d8f36e3ea41be353710b3f35733c4e57befbca82f58c77ee944"}
Jul 28 21:08:45.196183 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:45.196113    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" podStartSLOduration=1.196098558 podStartE2EDuration="1.196098558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:08:45.194626324 +0000 UTC m=+1419.593683010" watchObservedRunningTime="2026-07-28 21:08:45.196098558 +0000 UTC m=+1419.595155222"
Jul 28 21:08:53.522754 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.522708    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:08:53.526946 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.526921    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.529058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.529042    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f4f83f49-kserve-self-signed-certs\""
Jul 28 21:08:53.535862 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.535843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:08:53.607942 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.607911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.608102 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.607959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.608102 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.607983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.608102 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.608059    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnb5b\" (UniqueName: \"kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.608102 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.608096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.608300 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.608114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708659 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708836 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708673    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708836 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708836 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnb5b\" (UniqueName: \"kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708836 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.708836 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.708767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.709124 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.709101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.709225 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.709208    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.709304 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.709282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.711148 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.711117    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.711398 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.711383    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.715684 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.715661    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnb5b\" (UniqueName: \"kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b\") pod \"e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") " pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.838234 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.838129    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:08:53.972776 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:53.972755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:08:53.975006 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:08:53.974981    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf86e2fb_8146_4478_8206_38a3f9822c82.slice/crio-7ff291d943f5b3cf95dde1b416b2055caf09753555338ea23bdda12f455f37d6 WatchSource:0}: Error finding container 7ff291d943f5b3cf95dde1b416b2055caf09753555338ea23bdda12f455f37d6: Status 404 returned error can't find the container with id 7ff291d943f5b3cf95dde1b416b2055caf09753555338ea23bdda12f455f37d6
Jul 28 21:08:54.229452 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.229413    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" event={"ID":"bf86e2fb-8146-4478-8206-38a3f9822c82","Type":"ContainerStarted","Data":"06ecc9eb8613e1de4d51e9cfcf4bc57561266050434fe7e2d1f79707156952c4"}
Jul 28 21:08:54.229592 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.229460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" event={"ID":"bf86e2fb-8146-4478-8206-38a3f9822c82","Type":"ContainerStarted","Data":"7ff291d943f5b3cf95dde1b416b2055caf09753555338ea23bdda12f455f37d6"}
Jul 28 21:08:54.245830 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.245777    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" podStartSLOduration=1.245761903 podStartE2EDuration="1.245761903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:08:54.244656665 +0000 UTC m=+1428.643713329" watchObservedRunningTime="2026-07-28 21:08:54.245761903 +0000 UTC m=+1428.644818596"
Jul 28 21:08:54.449606 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.449527    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:54.449606 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.449566    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:54.461787 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:54.461767    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:55.246241 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:55.246214    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:08:56.452317 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:08:56.452284    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:09:03.839104 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:03.839065    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:03.839714 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:03.839124    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:03.851392 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:03.851368    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:04.293634 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:04.293597    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:23.277767 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:23.277729    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:09:24.260962 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:24.260928    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:09:25.344894 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:25.344858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:09:37.939683 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:37.939650    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:09:38.302988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.302949    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" podUID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" containerName="main" containerID="cri-o://cea1be478d88329a39040629449f1e20434ffc00b8f7e2bdf4ccb7ea8d1fc00c" gracePeriod=45
Jul 28 21:09:38.468009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.467976    2567 generic.go:356] "Generic (PLEG): container finished" podID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" containerID="cea1be478d88329a39040629449f1e20434ffc00b8f7e2bdf4ccb7ea8d1fc00c" exitCode=0
Jul 28 21:09:38.468184 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.468164    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" event={"ID":"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6","Type":"ContainerDied","Data":"cea1be478d88329a39040629449f1e20434ffc00b8f7e2bdf4ccb7ea8d1fc00c"}
Jul 28 21:09:38.548413 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.548388    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:09:38.700501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700419    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pdlq\" (UniqueName: \"kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700460    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700476    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700765 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700680    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700765 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700717    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:38.700765 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700752    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700920 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700805    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache\") pod \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\" (UID: \"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6\") "
Jul 28 21:09:38.700998 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.700973    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home" (OuterVolumeSpecName: "home") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:38.701085 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.701062    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:38.701226 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.701205    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-home\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:38.701297 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.701231    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-model-cache\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:38.701297 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.701247    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tmp-dir\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:38.702879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.702854    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:09:38.702879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.702860    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm" (OuterVolumeSpecName: "dshm") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:38.703033 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.702963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq" (OuterVolumeSpecName: "kube-api-access-5pdlq") pod "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" (UID: "a4f68590-23f1-4cf9-97e7-e6f89ccf2df6"). InnerVolumeSpecName "kube-api-access-5pdlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:09:38.801873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.801842    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pdlq\" (UniqueName: \"kubernetes.io/projected/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-kube-api-access-5pdlq\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:38.801873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.801867    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-dshm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:38.801873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:38.801879    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6-tls-certs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.280571 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.280526    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" podUID="bf86e2fb-8146-4478-8206-38a3f9822c82" containerName="main" containerID="cri-o://06ecc9eb8613e1de4d51e9cfcf4bc57561266050434fe7e2d1f79707156952c4" gracePeriod=45
Jul 28 21:09:39.482646 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.477345    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw" event={"ID":"a4f68590-23f1-4cf9-97e7-e6f89ccf2df6","Type":"ContainerDied","Data":"67b813ccd4d73d8f36e3ea41be353710b3f35733c4e57befbca82f58c77ee944"}
Jul 28 21:09:39.482646 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.477408    2567 scope.go:117] "RemoveContainer" containerID="cea1be478d88329a39040629449f1e20434ffc00b8f7e2bdf4ccb7ea8d1fc00c"
Jul 28 21:09:39.482646 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.477608    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"
Jul 28 21:09:39.484677 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.484645    2567 generic.go:356] "Generic (PLEG): container finished" podID="bf86e2fb-8146-4478-8206-38a3f9822c82" containerID="06ecc9eb8613e1de4d51e9cfcf4bc57561266050434fe7e2d1f79707156952c4" exitCode=0
Jul 28 21:09:39.484821 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.484686    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" event={"ID":"bf86e2fb-8146-4478-8206-38a3f9822c82","Type":"ContainerDied","Data":"06ecc9eb8613e1de4d51e9cfcf4bc57561266050434fe7e2d1f79707156952c4"}
Jul 28 21:09:39.504187 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.504160    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:09:39.507516 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.507485    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f4f83f49-kserve-6486f98447-khpcw"]
Jul 28 21:09:39.538263 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.538219    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:39.710672 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710647    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.710853 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710686    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.710853 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710724    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.710853 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710774    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnb5b\" (UniqueName: \"kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.710853 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710803    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.711071 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache\") pod \"bf86e2fb-8146-4478-8206-38a3f9822c82\" (UID: \"bf86e2fb-8146-4478-8206-38a3f9822c82\") "
Jul 28 21:09:39.711071 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710917    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home" (OuterVolumeSpecName: "home") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:39.711071 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.710953    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:39.711258 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.711157    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-home\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.711258 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.711194    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:39.711258 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.711202    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-tmp-dir\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.712925 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.712900    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:09:39.713036 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.712939    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b" (OuterVolumeSpecName: "kube-api-access-jnb5b") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "kube-api-access-jnb5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:09:39.713036 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.712964    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm" (OuterVolumeSpecName: "dshm") pod "bf86e2fb-8146-4478-8206-38a3f9822c82" (UID: "bf86e2fb-8146-4478-8206-38a3f9822c82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:09:39.811618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.811551    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf86e2fb-8146-4478-8206-38a3f9822c82-tls-certs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.811618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.811576    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnb5b\" (UniqueName: \"kubernetes.io/projected/bf86e2fb-8146-4478-8206-38a3f9822c82-kube-api-access-jnb5b\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.811618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.811585    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-dshm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:39.811618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:39.811594    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf86e2fb-8146-4478-8206-38a3f9822c82-model-cache\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:09:40.076287 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.076212    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" path="/var/lib/kubelet/pods/a4f68590-23f1-4cf9-97e7-e6f89ccf2df6/volumes"
Jul 28 21:09:40.493329 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.493247    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"
Jul 28 21:09:40.493329 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.493281    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x" event={"ID":"bf86e2fb-8146-4478-8206-38a3f9822c82","Type":"ContainerDied","Data":"7ff291d943f5b3cf95dde1b416b2055caf09753555338ea23bdda12f455f37d6"}
Jul 28 21:09:40.493329 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.493323    2567 scope.go:117] "RemoveContainer" containerID="06ecc9eb8613e1de4d51e9cfcf4bc57561266050434fe7e2d1f79707156952c4"
Jul 28 21:09:40.512040 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.512015    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:09:40.514492 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:40.514471    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f4f83f49-kserve-5848ff84dc-t745x"]
Jul 28 21:09:42.076697 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:42.076666    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf86e2fb-8146-4478-8206-38a3f9822c82" path="/var/lib/kubelet/pods/bf86e2fb-8146-4478-8206-38a3f9822c82/volumes"
Jul 28 21:09:47.844311 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:47.844280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:09:57.444511 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:09:57.444475    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:10:11.536330 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:10:11.536298    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:10:20.445267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:10:20.445235    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:10:30.949014 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:10:30.948934    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:10:39.046941 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:10:39.046895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:10:48.741581 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:10:48.741548    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:02.148534 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:02.148505    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:11.243925 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:11.243890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:20.946220 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:20.946188    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:29.943027 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:29.942991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:40.737621 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:40.737584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:11:49.440357 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:11:49.440318    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:01.541237 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:01.541163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:13.345968 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:13.345935    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:25.841258 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:25.841226    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:38.949541 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:38.949508    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:48.838885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:48.838851    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:12:58.139188 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:12:58.139157    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:13:08.852232 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:08.852196    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:13:30.770963 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.770887    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"]
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771303    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" containerName="main"
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771316    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" containerName="main"
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771333    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf86e2fb-8146-4478-8206-38a3f9822c82" containerName="main"
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771338    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf86e2fb-8146-4478-8206-38a3f9822c82" containerName="main"
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771411    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf86e2fb-8146-4478-8206-38a3f9822c82" containerName="main"
Jul 28 21:13:30.771448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.771424    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4f68590-23f1-4cf9-97e7-e6f89ccf2df6" containerName="main"
Jul 28 21:13:30.774481 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.774464    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:30.776575 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.776550    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:13:30.776709 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.776590    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:13:30.781843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.781817    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"]
Jul 28 21:13:30.800865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.800833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b71fef06-rh69k\" (UID: \"30b26b9f-a872-4607-b7ad-3c7d411e6c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:30.902105 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.902078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b71fef06-rh69k\" (UID: \"30b26b9f-a872-4607-b7ad-3c7d411e6c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:30.902665 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:30.902648    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b71fef06-rh69k\" (UID: \"30b26b9f-a872-4607-b7ad-3c7d411e6c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:31.086441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:31.086368    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:31.220845 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:31.220827    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"]
Jul 28 21:13:31.224272 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:13:31.224241    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b26b9f_a872_4607_b7ad_3c7d411e6c23.slice/crio-60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2 WatchSource:0}: Error finding container 60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2: Status 404 returned error can't find the container with id 60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2
Jul 28 21:13:31.226247 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:31.226232    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:13:31.717456 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:31.717416    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k" event={"ID":"30b26b9f-a872-4607-b7ad-3c7d411e6c23","Type":"ContainerStarted","Data":"60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2"}
Jul 28 21:13:34.734006 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:34.733966    2567 generic.go:356] "Generic (PLEG): container finished" podID="30b26b9f-a872-4607-b7ad-3c7d411e6c23" containerID="0faff8a6dfd192155d2c5006ad533bde1dd3dd250fb6e60c66064bb969008cbf" exitCode=0
Jul 28 21:13:34.734401 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:34.734021    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k" event={"ID":"30b26b9f-a872-4607-b7ad-3c7d411e6c23","Type":"ContainerDied","Data":"0faff8a6dfd192155d2c5006ad533bde1dd3dd250fb6e60c66064bb969008cbf"}
Jul 28 21:13:35.869123 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:35.869102    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:35.942977 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:35.942947    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca\") pod \"30b26b9f-a872-4607-b7ad-3c7d411e6c23\" (UID: \"30b26b9f-a872-4607-b7ad-3c7d411e6c23\") "
Jul 28 21:13:35.943299 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:35.943276    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30b26b9f-a872-4607-b7ad-3c7d411e6c23" (UID: "30b26b9f-a872-4607-b7ad-3c7d411e6c23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:13:36.043708 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:36.043679    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b26b9f-a872-4607-b7ad-3c7d411e6c23-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:13:36.745289 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:36.745256    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"
Jul 28 21:13:36.745289 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:36.745273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k" event={"ID":"30b26b9f-a872-4607-b7ad-3c7d411e6c23","Type":"ContainerDied","Data":"60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2"}
Jul 28 21:13:36.745501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:36.745317    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60498599d6abf297e59712489c8d55277a5e97f44defd59bcd06ceb3d67eeab2"
Jul 28 21:13:46.034814 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:46.034768    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"]
Jul 28 21:13:46.039045 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:46.039019    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b71fef06-rh69k"]
Jul 28 21:13:46.076251 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:46.076215    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30b26b9f-a872-4607-b7ad-3c7d411e6c23" path="/var/lib/kubelet/pods/30b26b9f-a872-4607-b7ad-3c7d411e6c23/volumes"
Jul 28 21:13:49.448219 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:49.448185    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:13:59.747171 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:13:59.747124    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:14:14.570709 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.570675    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"]
Jul 28 21:14:14.571117 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.571058    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30b26b9f-a872-4607-b7ad-3c7d411e6c23" containerName="revoke-keys"
Jul 28 21:14:14.571117 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.571070    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="30b26b9f-a872-4607-b7ad-3c7d411e6c23" containerName="revoke-keys"
Jul 28 21:14:14.571218 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.571125    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="30b26b9f-a872-4607-b7ad-3c7d411e6c23" containerName="revoke-keys"
Jul 28 21:14:14.577211 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.577188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:14.579324 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.579298    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:14:14.579471 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.579298    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:14:14.580569 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.580544    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"]
Jul 28 21:14:14.585462 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.585434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p\" (UID: \"eb704b0e-da6a-46cf-846a-93a1a739d569\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:14.686429 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.686397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p\" (UID: \"eb704b0e-da6a-46cf-846a-93a1a739d569\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:14.686957 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.686938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p\" (UID: \"eb704b0e-da6a-46cf-846a-93a1a739d569\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:14.890909 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:14.890831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:15.026841 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:15.026815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"]
Jul 28 21:14:15.029403 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:14:15.029377    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb704b0e_da6a_46cf_846a_93a1a739d569.slice/crio-4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698 WatchSource:0}: Error finding container 4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698: Status 404 returned error can't find the container with id 4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698
Jul 28 21:14:15.953657 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:15.953625    2567 generic.go:356] "Generic (PLEG): container finished" podID="eb704b0e-da6a-46cf-846a-93a1a739d569" containerID="7c3d7bd2eec610f1bf0c2bebfe699937e5367c46a212308ecab9ddeba45d9af1" exitCode=0
Jul 28 21:14:15.954058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:15.953711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p" event={"ID":"eb704b0e-da6a-46cf-846a-93a1a739d569","Type":"ContainerDied","Data":"7c3d7bd2eec610f1bf0c2bebfe699937e5367c46a212308ecab9ddeba45d9af1"}
Jul 28 21:14:15.954058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:15.953758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p" event={"ID":"eb704b0e-da6a-46cf-846a-93a1a739d569","Type":"ContainerStarted","Data":"4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698"}
Jul 28 21:14:17.087179 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.087155    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:17.107501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.107399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca\") pod \"eb704b0e-da6a-46cf-846a-93a1a739d569\" (UID: \"eb704b0e-da6a-46cf-846a-93a1a739d569\") "
Jul 28 21:14:17.108225 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.108081    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb704b0e-da6a-46cf-846a-93a1a739d569" (UID: "eb704b0e-da6a-46cf-846a-93a1a739d569"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:14:17.209395 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.209318    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb704b0e-da6a-46cf-846a-93a1a739d569-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:14:17.965692 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.965661    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"
Jul 28 21:14:17.965863 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.965660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p" event={"ID":"eb704b0e-da6a-46cf-846a-93a1a739d569","Type":"ContainerDied","Data":"4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698"}
Jul 28 21:14:17.965863 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:17.965816    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e5aa366d95d10c839cc45724126f8ef5eba742c39ca4a5e03dda9b53bb30698"
Jul 28 21:14:21.013699 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:21.013673    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"]
Jul 28 21:14:21.015708 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:21.015682    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9fd4a294-8cl9p"]
Jul 28 21:14:22.076671 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:22.076635    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb704b0e-da6a-46cf-846a-93a1a739d569" path="/var/lib/kubelet/pods/eb704b0e-da6a-46cf-846a-93a1a739d569/volumes"
Jul 28 21:14:24.765913 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:24.765884    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:14:34.847565 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:34.847530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:14:50.473509 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.473471    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"]
Jul 28 21:14:50.474087 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.474065    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb704b0e-da6a-46cf-846a-93a1a739d569" containerName="revoke-keys"
Jul 28 21:14:50.474087 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.474083    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb704b0e-da6a-46cf-846a-93a1a739d569" containerName="revoke-keys"
Jul 28 21:14:50.474249 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.474199    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb704b0e-da6a-46cf-846a-93a1a739d569" containerName="revoke-keys"
Jul 28 21:14:50.477215 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.477196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:50.479251 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.479230    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:14:50.479348 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.479265    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:14:50.483018 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.482998    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"]
Jul 28 21:14:50.606496 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.606465    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c452994-m4j7f\" (UID: \"8bafdef7-1200-4c9b-aebe-e97e1acaa368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:50.707101 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.707069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c452994-m4j7f\" (UID: \"8bafdef7-1200-4c9b-aebe-e97e1acaa368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:50.707676 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.707656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c452994-m4j7f\" (UID: \"8bafdef7-1200-4c9b-aebe-e97e1acaa368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:50.788998 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.788973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:50.934236 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:50.931347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"]
Jul 28 21:14:51.132846 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:51.132764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f" event={"ID":"8bafdef7-1200-4c9b-aebe-e97e1acaa368","Type":"ContainerStarted","Data":"6a5dddeee9c533b7e598c48d2b341cef51fe5bae7ebce6c23c84a985839f52c3"}
Jul 28 21:14:52.139214 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:52.139177    2567 generic.go:356] "Generic (PLEG): container finished" podID="8bafdef7-1200-4c9b-aebe-e97e1acaa368" containerID="6a41fac76f83ae22fb25a66d9e7acd1a7274b57d78118bdf93cff89cee190114" exitCode=0
Jul 28 21:14:52.139593 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:52.139263    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f" event={"ID":"8bafdef7-1200-4c9b-aebe-e97e1acaa368","Type":"ContainerDied","Data":"6a41fac76f83ae22fb25a66d9e7acd1a7274b57d78118bdf93cff89cee190114"}
Jul 28 21:14:53.274230 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:53.274210    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:53.429745 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:53.429649    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca\") pod \"8bafdef7-1200-4c9b-aebe-e97e1acaa368\" (UID: \"8bafdef7-1200-4c9b-aebe-e97e1acaa368\") "
Jul 28 21:14:53.429973 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:53.429950    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8bafdef7-1200-4c9b-aebe-e97e1acaa368" (UID: "8bafdef7-1200-4c9b-aebe-e97e1acaa368"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:14:53.531265 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:53.531223    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafdef7-1200-4c9b-aebe-e97e1acaa368-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:14:54.152039 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:54.152011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f" event={"ID":"8bafdef7-1200-4c9b-aebe-e97e1acaa368","Type":"ContainerDied","Data":"6a5dddeee9c533b7e598c48d2b341cef51fe5bae7ebce6c23c84a985839f52c3"}
Jul 28 21:14:54.152039 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:54.152039    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"
Jul 28 21:14:54.152263 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:54.152055    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a5dddeee9c533b7e598c48d2b341cef51fe5bae7ebce6c23c84a985839f52c3"
Jul 28 21:14:55.353749 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:55.353717    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"]
Jul 28 21:14:55.357630 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:55.357605    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c452994-m4j7f"]
Jul 28 21:14:56.081098 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:56.081063    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bafdef7-1200-4c9b-aebe-e97e1acaa368" path="/var/lib/kubelet/pods/8bafdef7-1200-4c9b-aebe-e97e1acaa368/volumes"
Jul 28 21:14:59.041379 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:14:59.041300    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:15:00.134967 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.134939    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:00.135445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.135339    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bafdef7-1200-4c9b-aebe-e97e1acaa368" containerName="revoke-keys"
Jul 28 21:15:00.135445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.135352    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bafdef7-1200-4c9b-aebe-e97e1acaa368" containerName="revoke-keys"
Jul 28 21:15:00.135445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.135438    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bafdef7-1200-4c9b-aebe-e97e1acaa368" containerName="revoke-keys"
Jul 28 21:15:00.139007 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.138987    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"]
Jul 28 21:15:00.139117 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.139098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"
Jul 28 21:15:00.141353 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.141337    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:15:00.142203 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.142190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"
Jul 28 21:15:00.148474 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.148455    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:00.151535 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.151515    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"
Jul 28 21:15:00.151841 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.151821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"]
Jul 28 21:15:00.160248 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.160232    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"
Jul 28 21:15:00.307643 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.307601    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:00.309216 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:15:00.309187    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a17e82_c931_48b1_95c9_78296c525360.slice/crio-25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc WatchSource:0}: Error finding container 25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc: Status 404 returned error can't find the container with id 25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc
Jul 28 21:15:00.330689 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:00.328175    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"]
Jul 28 21:15:00.331293 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:15:00.331254    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce35fc1_0e4c_4e59_9ef4_59ac43d6029a.slice/crio-a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98 WatchSource:0}: Error finding container a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98: Status 404 returned error can't find the container with id a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98
Jul 28 21:15:01.199791 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.199711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8" event={"ID":"9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a","Type":"ContainerStarted","Data":"c974c2b5eb9faa44050ccb871d44e18db3eeda797c1bb5fab14a74b497ed77ae"}
Jul 28 21:15:01.200173 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.199802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8" event={"ID":"9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a","Type":"ContainerStarted","Data":"a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98"}
Jul 28 21:15:01.201202 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.201178    2567 generic.go:356] "Generic (PLEG): container finished" podID="c8a17e82-c931-48b1-95c9-78296c525360" containerID="b21024f9a5b26d9fafc487b884086acd08f17330cef89232f8ef6018f0a5fb25" exitCode=7
Jul 28 21:15:01.201312 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.201217    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerDied","Data":"b21024f9a5b26d9fafc487b884086acd08f17330cef89232f8ef6018f0a5fb25"}
Jul 28 21:15:01.201312 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.201276    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerStarted","Data":"25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc"}
Jul 28 21:15:01.201445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.201433    2567 scope.go:117] "RemoveContainer" containerID="b21024f9a5b26d9fafc487b884086acd08f17330cef89232f8ef6018f0a5fb25"
Jul 28 21:15:01.215663 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:01.215626    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8" podStartSLOduration=1.215614099 podStartE2EDuration="1.215614099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:15:01.213215266 +0000 UTC m=+1795.612271931" watchObservedRunningTime="2026-07-28 21:15:01.215614099 +0000 UTC m=+1795.614670765"
Jul 28 21:15:02.208291 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.208253    2567 generic.go:356] "Generic (PLEG): container finished" podID="9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a" containerID="c974c2b5eb9faa44050ccb871d44e18db3eeda797c1bb5fab14a74b497ed77ae" exitCode=0
Jul 28 21:15:02.208735 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.208333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8" event={"ID":"9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a","Type":"ContainerDied","Data":"c974c2b5eb9faa44050ccb871d44e18db3eeda797c1bb5fab14a74b497ed77ae"}
Jul 28 21:15:02.210062 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.210042    2567 generic.go:356] "Generic (PLEG): container finished" podID="c8a17e82-c931-48b1-95c9-78296c525360" containerID="d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d" exitCode=7
Jul 28 21:15:02.210197 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.210072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerDied","Data":"d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d"}
Jul 28 21:15:02.210197 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.210127    2567 scope.go:117] "RemoveContainer" containerID="b21024f9a5b26d9fafc487b884086acd08f17330cef89232f8ef6018f0a5fb25"
Jul 28 21:15:02.210321 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:02.210308    2567 scope.go:117] "RemoveContainer" containerID="d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d"
Jul 28 21:15:02.210562 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:15:02.210541    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8_odh-ai-gateway-infra(c8a17e82-c931-48b1-95c9-78296c525360)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" podUID="c8a17e82-c931-48b1-95c9-78296c525360"
Jul 28 21:15:03.216975 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:03.216952    2567 scope.go:117] "RemoveContainer" containerID="d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d"
Jul 28 21:15:03.217405 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:15:03.217197    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8_odh-ai-gateway-infra(c8a17e82-c931-48b1-95c9-78296c525360)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" podUID="c8a17e82-c931-48b1-95c9-78296c525360"
Jul 28 21:15:03.353214 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:03.353189    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"
Jul 28 21:15:04.222903 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:04.222869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8" event={"ID":"9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a","Type":"ContainerDied","Data":"a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98"}
Jul 28 21:15:04.222903 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:04.222902    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754555-lscm8"
Jul 28 21:15:04.223427 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:04.222920    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a71462339d97e5f5fd123fe9bc4bddc4fb3447753590b28834c01860b5589f98"
Jul 28 21:15:09.191350 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:09.191311    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:15:18.070696 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:18.070658    2567 scope.go:117] "RemoveContainer" containerID="d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d"
Jul 28 21:15:18.299181 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:18.299125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerStarted","Data":"27d68c1a9e7e4c675b9afa004b9031149ede3a0322134a9d6afc92dfb3a5ee16"}
Jul 28 21:15:18.315045 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:18.315004    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" podStartSLOduration=18.314990565 podStartE2EDuration="18.314990565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:15:18.31261753 +0000 UTC m=+1812.711674195" watchObservedRunningTime="2026-07-28 21:15:18.314990565 +0000 UTC m=+1812.714047255"
Jul 28 21:15:19.305751 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:19.305719    2567 generic.go:356] "Generic (PLEG): container finished" podID="c8a17e82-c931-48b1-95c9-78296c525360" containerID="27d68c1a9e7e4c675b9afa004b9031149ede3a0322134a9d6afc92dfb3a5ee16" exitCode=0
Jul 28 21:15:19.306182 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:19.305797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerDied","Data":"27d68c1a9e7e4c675b9afa004b9031149ede3a0322134a9d6afc92dfb3a5ee16"}
Jul 28 21:15:19.306182 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:19.305855    2567 scope.go:117] "RemoveContainer" containerID="d70bc313023b923550262d64ff40bf9067a8af27591dfb1ea6771bc3b011460d"
Jul 28 21:15:19.328222 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:19.328195    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:20.443048 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:20.443028    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"
Jul 28 21:15:21.319444 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:21.319406    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"
Jul 28 21:15:21.319638 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:21.319449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8" event={"ID":"c8a17e82-c931-48b1-95c9-78296c525360","Type":"ContainerDied","Data":"25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc"}
Jul 28 21:15:21.319638 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:21.319494    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25874017f84537fc07ee8f1c0e5bcea4d0e764145e901d7e18da3135c5b54fcc"
Jul 28 21:15:21.350248 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:21.350221    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:21.354259 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:21.354238    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-42733177-29754555-gznv8"]
Jul 28 21:15:22.076369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:22.076339    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8a17e82-c931-48b1-95c9-78296c525360" path="/var/lib/kubelet/pods/c8a17e82-c931-48b1-95c9-78296c525360/volumes"
Jul 28 21:15:24.698643 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.698604    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"]
Jul 28 21:15:24.699186 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699168    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.699279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699189    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.699279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699199    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.699279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699207    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.699279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699227    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a" containerName="cleanup"
Jul 28 21:15:24.699279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699235    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a" containerName="cleanup"
Jul 28 21:15:24.699527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699358    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a" containerName="cleanup"
Jul 28 21:15:24.699527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699369    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.699527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.699378    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:15:24.702569 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.702548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:24.704635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.704617    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:15:24.704719 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.704641    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:15:24.708336 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.708316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"]
Jul 28 21:15:24.793264 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.793222    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-42733177-mvjsl\" (UID: \"b10cca60-8bfd-4f62-84ea-f4c4ed90f344\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:24.894668 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.894632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-42733177-mvjsl\" (UID: \"b10cca60-8bfd-4f62-84ea-f4c4ed90f344\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:24.895275 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:24.895258    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-42733177-mvjsl\" (UID: \"b10cca60-8bfd-4f62-84ea-f4c4ed90f344\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:25.014234 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:25.014210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:25.152845 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:25.152823    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"]
Jul 28 21:15:25.154785 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:15:25.154760    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10cca60_8bfd_4f62_84ea_f4c4ed90f344.slice/crio-4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1 WatchSource:0}: Error finding container 4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1: Status 404 returned error can't find the container with id 4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1
Jul 28 21:15:25.343630 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:25.343565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl" event={"ID":"b10cca60-8bfd-4f62-84ea-f4c4ed90f344","Type":"ContainerStarted","Data":"4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1"}
Jul 28 21:15:26.351787 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:26.351753    2567 generic.go:356] "Generic (PLEG): container finished" podID="b10cca60-8bfd-4f62-84ea-f4c4ed90f344" containerID="afb046c041f9298cfd6eb436dd20749577a7df0f2dd66c9c41c8e83f0f3f3cfb" exitCode=0
Jul 28 21:15:26.352180 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:26.351807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl" event={"ID":"b10cca60-8bfd-4f62-84ea-f4c4ed90f344","Type":"ContainerDied","Data":"afb046c041f9298cfd6eb436dd20749577a7df0f2dd66c9c41c8e83f0f3f3cfb"}
Jul 28 21:15:27.486794 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:27.486765    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:27.620316 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:27.620231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca\") pod \"b10cca60-8bfd-4f62-84ea-f4c4ed90f344\" (UID: \"b10cca60-8bfd-4f62-84ea-f4c4ed90f344\") "
Jul 28 21:15:27.620624 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:27.620602    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b10cca60-8bfd-4f62-84ea-f4c4ed90f344" (UID: "b10cca60-8bfd-4f62-84ea-f4c4ed90f344"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:15:27.721514 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:27.721481    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10cca60-8bfd-4f62-84ea-f4c4ed90f344-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:15:28.363921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:28.363824    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"
Jul 28 21:15:28.363921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:28.363854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl" event={"ID":"b10cca60-8bfd-4f62-84ea-f4c4ed90f344","Type":"ContainerDied","Data":"4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1"}
Jul 28 21:15:28.363921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:28.363910    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bffada8fe9292c0edb141e9bcd02d7bab5381c75538ca2d146d162f8a1d89f1"
Jul 28 21:15:30.811974 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:30.811935    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"]
Jul 28 21:15:30.814555 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:30.814532    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-42733177-mvjsl"]
Jul 28 21:15:32.075952 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:32.075920    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10cca60-8bfd-4f62-84ea-f4c4ed90f344" path="/var/lib/kubelet/pods/b10cca60-8bfd-4f62-84ea-f4c4ed90f344/volumes"
Jul 28 21:15:34.538937 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:15:34.538901    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:16:58.771589 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:16:58.771503    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:17:11.889288 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:17:11.889254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:17:22.257116 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:17:22.257080    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:17:33.847256 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:17:33.847215    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:18:01.677213 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677123    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"]
Jul 28 21:18:01.677635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677547    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10cca60-8bfd-4f62-84ea-f4c4ed90f344" containerName="revoke-keys"
Jul 28 21:18:01.677635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677560    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10cca60-8bfd-4f62-84ea-f4c4ed90f344" containerName="revoke-keys"
Jul 28 21:18:01.677635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677583    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:18:01.677635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677589    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:18:01.677782 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677651    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10cca60-8bfd-4f62-84ea-f4c4ed90f344" containerName="revoke-keys"
Jul 28 21:18:01.677782 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.677660    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a17e82-c931-48b1-95c9-78296c525360" containerName="cleanup"
Jul 28 21:18:01.680714 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.680693    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:01.682951 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.682931    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:18:01.683051 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.682979    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:18:01.692753 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.692729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"]
Jul 28 21:18:01.753230 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.753199    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r\" (UID: \"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:01.853775 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.853735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r\" (UID: \"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:01.854347 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.854330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r\" (UID: \"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:01.991248 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:01.991178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:02.125929 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:02.125855    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"]
Jul 28 21:18:02.128605 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:18:02.128578    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43744c0d_0ab4_4bda_9a3b_fe04f10bb9d4.slice/crio-2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8 WatchSource:0}: Error finding container 2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8: Status 404 returned error can't find the container with id 2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8
Jul 28 21:18:02.209062 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:02.209032    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r" event={"ID":"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4","Type":"ContainerStarted","Data":"2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8"}
Jul 28 21:18:03.216441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:03.216349    2567 generic.go:356] "Generic (PLEG): container finished" podID="43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" containerID="40181795478ed6d36d34549c98a86b22a7c55c583eba586a0f4d122a90c661b9" exitCode=0
Jul 28 21:18:03.216441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:03.216406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r" event={"ID":"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4","Type":"ContainerDied","Data":"40181795478ed6d36d34549c98a86b22a7c55c583eba586a0f4d122a90c661b9"}
Jul 28 21:18:04.350960 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:04.350936    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:04.475798 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:04.475713    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca\") pod \"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4\" (UID: \"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4\") "
Jul 28 21:18:04.476047 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:04.476023    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" (UID: "43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:18:04.577330 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:04.577303    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:18:05.228708 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:05.228678    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"
Jul 28 21:18:05.228885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:05.228712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r" event={"ID":"43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4","Type":"ContainerDied","Data":"2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8"}
Jul 28 21:18:05.228885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:05.228763    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc509a69d24301e0560d6a4ca2cb7ed13fb25250718f43ad224089aa57f90b8"
Jul 28 21:18:12.461887 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:12.461855    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"]
Jul 28 21:18:12.464496 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:12.464473    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d0e37f3c-sct2r"]
Jul 28 21:18:14.076086 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:14.076051    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" path="/var/lib/kubelet/pods/43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4/volumes"
Jul 28 21:18:14.949427 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:14.949397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:18:49.451620 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:18:49.451584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:19:17.636661 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:17.636628    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:19:38.647072 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:38.647034    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:19:43.240713 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.240678    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"]
Jul 28 21:19:43.241174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.241009    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" containerName="revoke-keys"
Jul 28 21:19:43.241174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.241020    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" containerName="revoke-keys"
Jul 28 21:19:43.241174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.241087    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="43744c0d-0ab4-4bda-9a3b-fe04f10bb9d4" containerName="revoke-keys"
Jul 28 21:19:43.244217 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.244198    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:43.246495 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.246474    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:19:43.246495 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.246489    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:19:43.252982 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.252958    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"]
Jul 28 21:19:43.322000 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.321960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2dd70051-mdc86\" (UID: \"1b30cfc3-7208-424f-91af-7ac3231765c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:43.422837 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.422801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2dd70051-mdc86\" (UID: \"1b30cfc3-7208-424f-91af-7ac3231765c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:43.423451 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.423433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2dd70051-mdc86\" (UID: \"1b30cfc3-7208-424f-91af-7ac3231765c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:43.555093 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.555064    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:43.690601 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.690579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"]
Jul 28 21:19:43.692726 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:19:43.692695    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b30cfc3_7208_424f_91af_7ac3231765c6.slice/crio-aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9 WatchSource:0}: Error finding container aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9: Status 404 returned error can't find the container with id aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9
Jul 28 21:19:43.757609 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:43.757583    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86" event={"ID":"1b30cfc3-7208-424f-91af-7ac3231765c6","Type":"ContainerStarted","Data":"aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9"}
Jul 28 21:19:44.764149 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:44.764109    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b30cfc3-7208-424f-91af-7ac3231765c6" containerID="1801176ad83e9ff907f43ea83c3ac814a6f8d864c66792858e83fc70585697bd" exitCode=0
Jul 28 21:19:44.764511 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:44.764200    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86" event={"ID":"1b30cfc3-7208-424f-91af-7ac3231765c6","Type":"ContainerDied","Data":"1801176ad83e9ff907f43ea83c3ac814a6f8d864c66792858e83fc70585697bd"}
Jul 28 21:19:45.897880 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:45.897857    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:46.045026 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.044942    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca\") pod \"1b30cfc3-7208-424f-91af-7ac3231765c6\" (UID: \"1b30cfc3-7208-424f-91af-7ac3231765c6\") "
Jul 28 21:19:46.045366 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.045338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b30cfc3-7208-424f-91af-7ac3231765c6" (UID: "1b30cfc3-7208-424f-91af-7ac3231765c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:19:46.145921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.145892    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b30cfc3-7208-424f-91af-7ac3231765c6-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:19:46.775843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.775808    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"
Jul 28 21:19:46.776016 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.775860    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86" event={"ID":"1b30cfc3-7208-424f-91af-7ac3231765c6","Type":"ContainerDied","Data":"aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9"}
Jul 28 21:19:46.776016 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:46.775910    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa9bbdbdfa2586dea4d8396e2548637abbde17c3899fce3c4f4fffac3d1a43f9"
Jul 28 21:19:57.447487 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:57.447453    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:19:58.920451 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:58.920401    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"]
Jul 28 21:19:58.924763 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:19:58.924736    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2dd70051-mdc86"]
Jul 28 21:20:00.075542 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:20:00.075502    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b30cfc3-7208-424f-91af-7ac3231765c6" path="/var/lib/kubelet/pods/1b30cfc3-7208-424f-91af-7ac3231765c6/volumes"
Jul 28 21:20:06.272153 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:20:06.272117    2567 scope.go:117] "RemoveContainer" containerID="0faff8a6dfd192155d2c5006ad533bde1dd3dd250fb6e60c66064bb969008cbf"
Jul 28 21:20:08.149827 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:20:08.149793    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:20:54.551606 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:20:54.551570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:21:04.543043 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:21:04.543006    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:21:06.292574 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:21:06.292543    2567 scope.go:117] "RemoveContainer" containerID="6a41fac76f83ae22fb25a66d9e7acd1a7274b57d78118bdf93cff89cee190114"
Jul 28 21:21:06.301149 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:21:06.301116    2567 scope.go:117] "RemoveContainer" containerID="7c3d7bd2eec610f1bf0c2bebfe699937e5367c46a212308ecab9ddeba45d9af1"
Jul 28 21:21:55.648393 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:21:55.648354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:22:06.328353 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:06.328323    2567 scope.go:117] "RemoveContainer" containerID="afb046c041f9298cfd6eb436dd20749577a7df0f2dd66c9c41c8e83f0f3f3cfb"
Jul 28 21:22:06.336961 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:06.336940    2567 scope.go:117] "RemoveContainer" containerID="27d68c1a9e7e4c675b9afa004b9031149ede3a0322134a9d6afc92dfb3a5ee16"
Jul 28 21:22:28.441167 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:28.441065    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:22:55.038039 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.037999    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:22:55.038568 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.038531    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b30cfc3-7208-424f-91af-7ac3231765c6" containerName="revoke-keys"
Jul 28 21:22:55.038568 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.038551    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b30cfc3-7208-424f-91af-7ac3231765c6" containerName="revoke-keys"
Jul 28 21:22:55.038686 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.038667    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b30cfc3-7208-424f-91af-7ac3231765c6" containerName="revoke-keys"
Jul 28 21:22:55.041653 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.041634    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.046369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.043796    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d9fd50\"/\"default-dockercfg-hp2wf\""
Jul 28 21:22:55.046369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.044524    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d9fd50\"/\"test-model-d9fd50-kserve-self-signed-certs\""
Jul 28 21:22:55.046369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.044768    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d9fd50\"/\"kube-root-ca.crt\""
Jul 28 21:22:55.046369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.044967    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d9fd50\"/\"openshift-service-ca.crt\""
Jul 28 21:22:55.056398 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.056374    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:22:55.114771 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.114771 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.114988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.114988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.114988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfzcb\" (UniqueName: \"kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.114988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.114950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216243 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216391 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfzcb\" (UniqueName: \"kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216464 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216406    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216525 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216588 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216537    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216648 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216648 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216799 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.216901 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.216827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.218894 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.218878    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.218965 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.218950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.225019 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.225003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfzcb\" (UniqueName: \"kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb\") pod \"test-model-d9fd50-kserve-d589f9ccc-dxnm6\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") " pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.357811 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.357726    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:22:55.491009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.490986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:22:55.493203 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:22:55.493172    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bf58347_6809_4111_9104_571424b6cb2d.slice/crio-7916257b30eaf7a88c1d3c1a8da00c817e79c9cf7258297a284fd5f47b683d05 WatchSource:0}: Error finding container 7916257b30eaf7a88c1d3c1a8da00c817e79c9cf7258297a284fd5f47b683d05: Status 404 returned error can't find the container with id 7916257b30eaf7a88c1d3c1a8da00c817e79c9cf7258297a284fd5f47b683d05
Jul 28 21:22:55.800583 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.800548    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" event={"ID":"8bf58347-6809-4111-9104-571424b6cb2d","Type":"ContainerStarted","Data":"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"}
Jul 28 21:22:55.800750 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.800591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" event={"ID":"8bf58347-6809-4111-9104-571424b6cb2d","Type":"ContainerStarted","Data":"7916257b30eaf7a88c1d3c1a8da00c817e79c9cf7258297a284fd5f47b683d05"}
Jul 28 21:22:55.827241 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:55.827196    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" podStartSLOduration=0.827181113 podStartE2EDuration="827.181113ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:22:55.82545858 +0000 UTC m=+2270.224515245" watchObservedRunningTime="2026-07-28 21:22:55.827181113 +0000 UTC m=+2270.226237780"
Jul 28 21:22:59.046287 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:22:59.046252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:23:05.358832 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:23:05.358798    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:23:05.359302 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:23:05.358845    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:23:05.371516 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:23:05.371485    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:23:05.865660 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:23:05.865627    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:23:22.863921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:23:22.863888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:24:05.761907 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:05.761876    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:24:06.372623 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.372595    2567 scope.go:117] "RemoveContainer" containerID="40181795478ed6d36d34549c98a86b22a7c55c583eba586a0f4d122a90c661b9"
Jul 28 21:24:06.849521 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.849486    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"]
Jul 28 21:24:06.854074 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.854055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:06.856467 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.856441    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:24:06.856467 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.856459    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:24:06.860178 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.860127    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"]
Jul 28 21:24:06.922054 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:06.922027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb\" (UID: \"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:07.022743 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:07.022708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb\" (UID: \"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:07.023304 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:07.023288    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb\" (UID: \"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:07.165012 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:07.164917    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:07.305240 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:07.305215    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"]
Jul 28 21:24:07.307300 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:24:07.307277    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5c307e_5f40_47c0_82c1_8fc6c47a9b36.slice/crio-f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16 WatchSource:0}: Error finding container f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16: Status 404 returned error can't find the container with id f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16
Jul 28 21:24:08.193787 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:08.193741    2567 generic.go:356] "Generic (PLEG): container finished" podID="8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" containerID="ce62cdf94d21bba1a4d3f27e1f24f7efd445b8de640febefadaa719e08e20b8b" exitCode=0
Jul 28 21:24:08.194203 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:08.193829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb" event={"ID":"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36","Type":"ContainerDied","Data":"ce62cdf94d21bba1a4d3f27e1f24f7efd445b8de640febefadaa719e08e20b8b"}
Jul 28 21:24:08.194203 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:08.193896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb" event={"ID":"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36","Type":"ContainerStarted","Data":"f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16"}
Jul 28 21:24:09.323660 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:09.323638    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:09.339793 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:09.339755    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca\") pod \"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36\" (UID: \"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36\") "
Jul 28 21:24:09.340122 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:09.340095    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" (UID: "8a5c307e-5f40-47c0-82c1-8fc6c47a9b36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:24:09.441044 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:09.441012    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:10.205471 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:10.205441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb" event={"ID":"8a5c307e-5f40-47c0-82c1-8fc6c47a9b36","Type":"ContainerDied","Data":"f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16"}
Jul 28 21:24:10.205607 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:10.205489    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f29c7a7fb6d5d27d9d9d04a4ffa425661ac61966da70246b1eeecb4937deac16"
Jul 28 21:24:10.205607 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:10.205447    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"
Jul 28 21:24:10.843186 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:10.843151    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:24:17.763494 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:17.763447    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"]
Jul 28 21:24:17.765600 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:17.765581    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d9fd50-r4srb"]
Jul 28 21:24:18.076891 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:18.076817    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" path="/var/lib/kubelet/pods/8a5c307e-5f40-47c0-82c1-8fc6c47a9b36/volumes"
Jul 28 21:24:20.781475 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:20.781421    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" podUID="8bf58347-6809-4111-9104-571424b6cb2d" containerName="main" containerID="cri-o://e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1" gracePeriod=45
Jul 28 21:24:21.026386 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.026364    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:24:21.139344 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139266    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139344 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139303    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139344 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139342    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfzcb\" (UniqueName: \"kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139577 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139390    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139577 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139426    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139577 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139453    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home\") pod \"8bf58347-6809-4111-9104-571424b6cb2d\" (UID: \"8bf58347-6809-4111-9104-571424b6cb2d\") "
Jul 28 21:24:21.139577 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139511    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:24:21.139777 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139635    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:24:21.139777 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139765    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-tmp-dir\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.139888 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139786    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-model-cache\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.139888 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.139824    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home" (OuterVolumeSpecName: "home") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:24:21.141727 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.141707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:24:21.141812 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.141744    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm" (OuterVolumeSpecName: "dshm") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:24:21.141812 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.141799    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb" (OuterVolumeSpecName: "kube-api-access-bfzcb") pod "8bf58347-6809-4111-9104-571424b6cb2d" (UID: "8bf58347-6809-4111-9104-571424b6cb2d"). InnerVolumeSpecName "kube-api-access-bfzcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:24:21.240441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.240409    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-dshm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.240441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.240434    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf58347-6809-4111-9104-571424b6cb2d-home\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.240441 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.240443    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf58347-6809-4111-9104-571424b6cb2d-tls-certs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.240662 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.240453    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfzcb\" (UniqueName: \"kubernetes.io/projected/8bf58347-6809-4111-9104-571424b6cb2d-kube-api-access-bfzcb\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:21.264144 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.264109    2567 generic.go:356] "Generic (PLEG): container finished" podID="8bf58347-6809-4111-9104-571424b6cb2d" containerID="e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1" exitCode=0
Jul 28 21:24:21.264267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.264182    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"
Jul 28 21:24:21.264267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.264193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" event={"ID":"8bf58347-6809-4111-9104-571424b6cb2d","Type":"ContainerDied","Data":"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"}
Jul 28 21:24:21.264267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.264232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6" event={"ID":"8bf58347-6809-4111-9104-571424b6cb2d","Type":"ContainerDied","Data":"7916257b30eaf7a88c1d3c1a8da00c817e79c9cf7258297a284fd5f47b683d05"}
Jul 28 21:24:21.264267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.264248    2567 scope.go:117] "RemoveContainer" containerID="e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"
Jul 28 21:24:21.274000 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.273983    2567 scope.go:117] "RemoveContainer" containerID="e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"
Jul 28 21:24:21.274298 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:24:21.274280    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1\": container with ID starting with e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1 not found: ID does not exist" containerID="e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"
Jul 28 21:24:21.274375 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.274303    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1"} err="failed to get container status \"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1\": rpc error: code = NotFound desc = could not find container \"e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1\": container with ID starting with e863511beaa03f808ffad9cf7afe324d099841ede141326fb97872deafb32cc1 not found: ID does not exist"
Jul 28 21:24:21.285045 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.285023    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:24:21.288585 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:21.288567    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d9fd50/test-model-d9fd50-kserve-d589f9ccc-dxnm6"]
Jul 28 21:24:22.076335 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:22.076296    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf58347-6809-4111-9104-571424b6cb2d" path="/var/lib/kubelet/pods/8bf58347-6809-4111-9104-571424b6cb2d/volumes"
Jul 28 21:24:25.294651 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.294617    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"]
Jul 28 21:24:25.295009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.294969    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" containerName="revoke-keys"
Jul 28 21:24:25.295009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.294979    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" containerName="revoke-keys"
Jul 28 21:24:25.295009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.294989    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf58347-6809-4111-9104-571424b6cb2d" containerName="main"
Jul 28 21:24:25.295009 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.294994    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf58347-6809-4111-9104-571424b6cb2d" containerName="main"
Jul 28 21:24:25.295213 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.295047    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf58347-6809-4111-9104-571424b6cb2d" containerName="main"
Jul 28 21:24:25.295213 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.295057    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5c307e-5f40-47c0-82c1-8fc6c47a9b36" containerName="revoke-keys"
Jul 28 21:24:25.299519 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.299503    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:25.302427 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.302407    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:24:25.305838 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.305807    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:24:25.310404 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.310387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"]
Jul 28 21:24:25.375699 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.375668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt\" (UID: \"272e03f8-c3d2-4292-b146-7b5012a420de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:25.476565 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.476528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt\" (UID: \"272e03f8-c3d2-4292-b146-7b5012a420de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:25.477175 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.477128    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt\" (UID: \"272e03f8-c3d2-4292-b146-7b5012a420de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:25.609684 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.609595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:25.744841 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:25.744815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"]
Jul 28 21:24:25.747617 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:24:25.747594    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272e03f8_c3d2_4292_b146_7b5012a420de.slice/crio-291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2 WatchSource:0}: Error finding container 291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2: Status 404 returned error can't find the container with id 291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2
Jul 28 21:24:26.292967 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:26.292937    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt" event={"ID":"272e03f8-c3d2-4292-b146-7b5012a420de","Type":"ContainerStarted","Data":"9ddea44a76e5853a193aea3244659b1bf8df530919e90089bdb6ea392a73f4f1"}
Jul 28 21:24:26.293118 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:26.292989    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt" event={"ID":"272e03f8-c3d2-4292-b146-7b5012a420de","Type":"ContainerStarted","Data":"291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2"}
Jul 28 21:24:27.299665 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:27.299633    2567 generic.go:356] "Generic (PLEG): container finished" podID="272e03f8-c3d2-4292-b146-7b5012a420de" containerID="9ddea44a76e5853a193aea3244659b1bf8df530919e90089bdb6ea392a73f4f1" exitCode=0
Jul 28 21:24:27.300052 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:27.299682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt" event={"ID":"272e03f8-c3d2-4292-b146-7b5012a420de","Type":"ContainerDied","Data":"9ddea44a76e5853a193aea3244659b1bf8df530919e90089bdb6ea392a73f4f1"}
Jul 28 21:24:27.447374 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:27.447337    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:24:28.434497 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:28.434475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:28.503114 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:28.503084    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca\") pod \"272e03f8-c3d2-4292-b146-7b5012a420de\" (UID: \"272e03f8-c3d2-4292-b146-7b5012a420de\") "
Jul 28 21:24:28.503402 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:28.503380    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "272e03f8-c3d2-4292-b146-7b5012a420de" (UID: "272e03f8-c3d2-4292-b146-7b5012a420de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:24:28.604117 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:28.604054    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/272e03f8-c3d2-4292-b146-7b5012a420de-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:24:29.311200 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:29.311172    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"
Jul 28 21:24:29.311393 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:29.311196    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt" event={"ID":"272e03f8-c3d2-4292-b146-7b5012a420de","Type":"ContainerDied","Data":"291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2"}
Jul 28 21:24:29.311393 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:29.311257    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="291e27b3bd8217196557413440d1c096c4af5057a950ab14d15762a3b6e043c2"
Jul 28 21:24:35.912010 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:35.911968    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"]
Jul 28 21:24:35.914769 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:35.914748    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9bf2ee-n9qbt"]
Jul 28 21:24:36.076092 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:36.076053    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="272e03f8-c3d2-4292-b146-7b5012a420de" path="/var/lib/kubelet/pods/272e03f8-c3d2-4292-b146-7b5012a420de/volumes"
Jul 28 21:24:38.540893 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:38.540857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:24:50.246339 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:24:50.246304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:25:15.867844 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.867815    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:25:15.868283 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.868214    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272e03f8-c3d2-4292-b146-7b5012a420de" containerName="revoke-keys"
Jul 28 21:25:15.868283 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.868233    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="272e03f8-c3d2-4292-b146-7b5012a420de" containerName="revoke-keys"
Jul 28 21:25:15.868360 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.868307    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="272e03f8-c3d2-4292-b146-7b5012a420de" containerName="revoke-keys"
Jul 28 21:25:15.871243 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.871222    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:15.874269 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.874254    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sz5mx\""
Jul 28 21:25:15.874356 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.874337    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-074f3a-serving-cert\""
Jul 28 21:25:15.874453 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.874438    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:25:15.878228 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.878213    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 21:25:15.883460 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.883434    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:25:15.900620 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.900593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:15.900735 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.900641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqr8\" (UniqueName: \"kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:15.900800 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:15.900726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.001393 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.001360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqr8\" (UniqueName: \"kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.001553 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.001429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.001553 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.001493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.001666 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:25:16.001564    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-serving-cert: secret "maas-api-e2e-api-b-074f3a-serving-cert" not found
Jul 28 21:25:16.001666 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:25:16.001629    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls podName:667dfc2f-312d-47d2-8d24-5a12cf262841 nodeName:}" failed. No retries permitted until 2026-07-28 21:25:16.50161288 +0000 UTC m=+2410.900669523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls") pod "maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" (UID: "667dfc2f-312d-47d2-8d24-5a12cf262841") : secret "maas-api-e2e-api-b-074f3a-serving-cert" not found
Jul 28 21:25:16.002100 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.002078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.013739 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.013712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqr8\" (UniqueName: \"kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.506663 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.506629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.509145 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.509116    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") pod \"maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.781012 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.780984    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:16.918793 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.918770    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:25:16.920417 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:25:16.920391    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667dfc2f_312d_47d2_8d24_5a12cf262841.slice/crio-8cf045b67c5994f239e9992a6b0e93b6255eecfe15d13ff48c1f77aaa50b5d30 WatchSource:0}: Error finding container 8cf045b67c5994f239e9992a6b0e93b6255eecfe15d13ff48c1f77aaa50b5d30: Status 404 returned error can't find the container with id 8cf045b67c5994f239e9992a6b0e93b6255eecfe15d13ff48c1f77aaa50b5d30
Jul 28 21:25:16.921805 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:16.921785    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:25:17.584443 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:17.584398    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" event={"ID":"667dfc2f-312d-47d2-8d24-5a12cf262841","Type":"ContainerStarted","Data":"8cf045b67c5994f239e9992a6b0e93b6255eecfe15d13ff48c1f77aaa50b5d30"}
Jul 28 21:25:19.446938 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:19.446909    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:25:19.597687 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:19.597592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" event={"ID":"667dfc2f-312d-47d2-8d24-5a12cf262841","Type":"ContainerStarted","Data":"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"}
Jul 28 21:25:19.597850 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:19.597792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:19.615930 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:19.615883    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" podStartSLOduration=2.269883905 podStartE2EDuration="4.615870346s" totalImagesPullingTime="2.345986441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:25:19.614457172 +0000 UTC m=+2414.013513837" watchObservedRunningTime="2026-07-28 21:25:19.615870346 +0000 UTC m=+2414.014927010"
Jul 28 21:25:25.607849 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:25.607816    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:25:43.748297 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.748264    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"]
Jul 28 21:25:43.751572 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.751550    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:43.753844 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.753821    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:25:43.758781 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.758759    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"]
Jul 28 21:25:43.838870 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.838840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4\" (UID: \"a328ebdd-0777-4290-91e2-9463651ce628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:43.940212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.940176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4\" (UID: \"a328ebdd-0777-4290-91e2-9463651ce628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:43.940766 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:43.940749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4\" (UID: \"a328ebdd-0777-4290-91e2-9463651ce628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:44.062979 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:44.062934    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:44.193270 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:44.193242    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"]
Jul 28 21:25:44.195297 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:25:44.195272    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda328ebdd_0777_4290_91e2_9463651ce628.slice/crio-96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035 WatchSource:0}: Error finding container 96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035: Status 404 returned error can't find the container with id 96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035
Jul 28 21:25:44.731001 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:44.730915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4" event={"ID":"a328ebdd-0777-4290-91e2-9463651ce628","Type":"ContainerStarted","Data":"6e0597e6e35ac065a50689a4d530df4c06c795afab214b834446b20a8d48d9a7"}
Jul 28 21:25:44.731001 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:44.730962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4" event={"ID":"a328ebdd-0777-4290-91e2-9463651ce628","Type":"ContainerStarted","Data":"96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035"}
Jul 28 21:25:44.744122 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:44.744076    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4" podStartSLOduration=1.744062015 podStartE2EDuration="1.744062015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:25:44.743149899 +0000 UTC m=+2439.142206555" watchObservedRunningTime="2026-07-28 21:25:44.744062015 +0000 UTC m=+2439.143118658"
Jul 28 21:25:45.738153 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:45.738112    2567 generic.go:356] "Generic (PLEG): container finished" podID="a328ebdd-0777-4290-91e2-9463651ce628" containerID="6e0597e6e35ac065a50689a4d530df4c06c795afab214b834446b20a8d48d9a7" exitCode=0
Jul 28 21:25:45.738556 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:45.738170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4" event={"ID":"a328ebdd-0777-4290-91e2-9463651ce628","Type":"ContainerDied","Data":"6e0597e6e35ac065a50689a4d530df4c06c795afab214b834446b20a8d48d9a7"}
Jul 28 21:25:46.878989 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:46.878969    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:46.968945 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:46.968906    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca\") pod \"a328ebdd-0777-4290-91e2-9463651ce628\" (UID: \"a328ebdd-0777-4290-91e2-9463651ce628\") "
Jul 28 21:25:46.969308 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:46.969288    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a328ebdd-0777-4290-91e2-9463651ce628" (UID: "a328ebdd-0777-4290-91e2-9463651ce628"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:25:47.069783 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:47.069750    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a328ebdd-0777-4290-91e2-9463651ce628-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:25:47.749505 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:47.749475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"
Jul 28 21:25:47.749673 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:47.749514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4" event={"ID":"a328ebdd-0777-4290-91e2-9463651ce628","Type":"ContainerDied","Data":"96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035"}
Jul 28 21:25:47.749673 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:47.749562    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96dfc1f8091a7ac179a81fbdf17dcbd95decb34d318cbaa03d3dbd0299b77035"
Jul 28 21:25:59.457756 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:59.457720    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"]
Jul 28 21:25:59.460729 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:25:59.460705    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ee8397-ncxj4"]
Jul 28 21:26:00.076103 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:00.076073    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a328ebdd-0777-4290-91e2-9463651ce628" path="/var/lib/kubelet/pods/a328ebdd-0777-4290-91e2-9463651ce628/volumes"
Jul 28 21:26:01.445803 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:01.445768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:26:06.406013 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.405982    2567 scope.go:117] "RemoveContainer" containerID="1801176ad83e9ff907f43ea83c3ac814a6f8d864c66792858e83fc70585697bd"
Jul 28 21:26:06.567992 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.567961    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"]
Jul 28 21:26:06.568345 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.568331    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a328ebdd-0777-4290-91e2-9463651ce628" containerName="revoke-keys"
Jul 28 21:26:06.568345 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.568345    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a328ebdd-0777-4290-91e2-9463651ce628" containerName="revoke-keys"
Jul 28 21:26:06.568445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.568395    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a328ebdd-0777-4290-91e2-9463651ce628" containerName="revoke-keys"
Jul 28 21:26:06.575192 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.575161    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:06.577536 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.577237    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:26:06.577878 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.577848    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"]
Jul 28 21:26:06.637475 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.637446    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv\" (UID: \"8f32c5de-9065-40fc-ac8d-392081782007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:06.738473 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.738393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv\" (UID: \"8f32c5de-9065-40fc-ac8d-392081782007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:06.738947 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.738930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv\" (UID: \"8f32c5de-9065-40fc-ac8d-392081782007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:06.887974 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:06.887947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:07.025870 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:07.025841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"]
Jul 28 21:26:07.028895 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:26:07.028865    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f32c5de_9065_40fc_ac8d_392081782007.slice/crio-414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a WatchSource:0}: Error finding container 414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a: Status 404 returned error can't find the container with id 414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a
Jul 28 21:26:07.862043 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:07.862004    2567 generic.go:356] "Generic (PLEG): container finished" podID="8f32c5de-9065-40fc-ac8d-392081782007" containerID="36b92f2f35d567a0e60f308a9b61e5208f2f6e3ee61348a580ef636a875b3ac0" exitCode=0
Jul 28 21:26:07.862422 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:07.862089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv" event={"ID":"8f32c5de-9065-40fc-ac8d-392081782007","Type":"ContainerDied","Data":"36b92f2f35d567a0e60f308a9b61e5208f2f6e3ee61348a580ef636a875b3ac0"}
Jul 28 21:26:07.862422 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:07.862170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv" event={"ID":"8f32c5de-9065-40fc-ac8d-392081782007","Type":"ContainerStarted","Data":"414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a"}
Jul 28 21:26:09.003681 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.003658    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:09.059897 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.059871    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca\") pod \"8f32c5de-9065-40fc-ac8d-392081782007\" (UID: \"8f32c5de-9065-40fc-ac8d-392081782007\") "
Jul 28 21:26:09.060228 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.060208    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f32c5de-9065-40fc-ac8d-392081782007" (UID: "8f32c5de-9065-40fc-ac8d-392081782007"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:26:09.166796 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.161787    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f32c5de-9065-40fc-ac8d-392081782007-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:26:09.874590 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.874558    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"
Jul 28 21:26:09.874782 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.874608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv" event={"ID":"8f32c5de-9065-40fc-ac8d-392081782007","Type":"ContainerDied","Data":"414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a"}
Jul 28 21:26:09.874782 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:09.874696    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="414407353da2fb26b5fd2971407e456837878aecbe8c2d4cc0f95f140dfc3a3a"
Jul 28 21:26:22.000642 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.000607    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:26:22.001185 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.000888    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" podUID="667dfc2f-312d-47d2-8d24-5a12cf262841" containerName="maas-api" containerID="cri-o://5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488" gracePeriod=30
Jul 28 21:26:22.193287 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.193225    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"]
Jul 28 21:26:22.195566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.195541    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-074f3a-5kmpv"]
Jul 28 21:26:22.256977 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.256922    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:26:22.378693 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.378656    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca\") pod \"667dfc2f-312d-47d2-8d24-5a12cf262841\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") "
Jul 28 21:26:22.378865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.378716    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") pod \"667dfc2f-312d-47d2-8d24-5a12cf262841\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") "
Jul 28 21:26:22.378865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.378756    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nqr8\" (UniqueName: \"kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8\") pod \"667dfc2f-312d-47d2-8d24-5a12cf262841\" (UID: \"667dfc2f-312d-47d2-8d24-5a12cf262841\") "
Jul 28 21:26:22.379086 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.379059    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "667dfc2f-312d-47d2-8d24-5a12cf262841" (UID: "667dfc2f-312d-47d2-8d24-5a12cf262841"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:26:22.381029 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.380999    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "667dfc2f-312d-47d2-8d24-5a12cf262841" (UID: "667dfc2f-312d-47d2-8d24-5a12cf262841"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:26:22.381174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.381073    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8" (OuterVolumeSpecName: "kube-api-access-4nqr8") pod "667dfc2f-312d-47d2-8d24-5a12cf262841" (UID: "667dfc2f-312d-47d2-8d24-5a12cf262841"). InnerVolumeSpecName "kube-api-access-4nqr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:26:22.480279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.480241    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667dfc2f-312d-47d2-8d24-5a12cf262841-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:26:22.480279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.480271    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667dfc2f-312d-47d2-8d24-5a12cf262841-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:26:22.480279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.480280    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nqr8\" (UniqueName: \"kubernetes.io/projected/667dfc2f-312d-47d2-8d24-5a12cf262841-kube-api-access-4nqr8\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:26:22.950972 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.950926    2567 generic.go:356] "Generic (PLEG): container finished" podID="667dfc2f-312d-47d2-8d24-5a12cf262841" containerID="5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488" exitCode=0
Jul 28 21:26:22.950972 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.950976    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"
Jul 28 21:26:22.951236 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.951020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" event={"ID":"667dfc2f-312d-47d2-8d24-5a12cf262841","Type":"ContainerDied","Data":"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"}
Jul 28 21:26:22.951236 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.951079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l" event={"ID":"667dfc2f-312d-47d2-8d24-5a12cf262841","Type":"ContainerDied","Data":"8cf045b67c5994f239e9992a6b0e93b6255eecfe15d13ff48c1f77aaa50b5d30"}
Jul 28 21:26:22.951236 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.951103    2567 scope.go:117] "RemoveContainer" containerID="5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"
Jul 28 21:26:22.964032 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.964007    2567 scope.go:117] "RemoveContainer" containerID="5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"
Jul 28 21:26:22.964372 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:26:22.964352    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488\": container with ID starting with 5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488 not found: ID does not exist" containerID="5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"
Jul 28 21:26:22.964435 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.964379    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488"} err="failed to get container status \"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488\": rpc error: code = NotFound desc = could not find container \"5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488\": container with ID starting with 5bf308706a5c626764ed7ba152514d1effe3a42c494f9c65920c1ce44d7b2488 not found: ID does not exist"
Jul 28 21:26:22.980540 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.980514    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:26:22.984773 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:22.984754    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-074f3a-7c8bdbf5df-rqv7l"]
Jul 28 21:26:24.075805 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:24.075774    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="667dfc2f-312d-47d2-8d24-5a12cf262841" path="/var/lib/kubelet/pods/667dfc2f-312d-47d2-8d24-5a12cf262841/volumes"
Jul 28 21:26:24.076194 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:24.076077    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f32c5de-9065-40fc-ac8d-392081782007" path="/var/lib/kubelet/pods/8f32c5de-9065-40fc-ac8d-392081782007/volumes"
Jul 28 21:26:26.851453 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:26.851421    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:26:33.246781 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.246743    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:26:33.247347 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247321    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667dfc2f-312d-47d2-8d24-5a12cf262841" containerName="maas-api"
Jul 28 21:26:33.247347 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="667dfc2f-312d-47d2-8d24-5a12cf262841" containerName="maas-api"
Jul 28 21:26:33.247532 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247364    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f32c5de-9065-40fc-ac8d-392081782007" containerName="revoke-keys"
Jul 28 21:26:33.247532 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247374    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f32c5de-9065-40fc-ac8d-392081782007" containerName="revoke-keys"
Jul 28 21:26:33.247532 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247493    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="667dfc2f-312d-47d2-8d24-5a12cf262841" containerName="maas-api"
Jul 28 21:26:33.247532 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.247506    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f32c5de-9065-40fc-ac8d-392081782007" containerName="revoke-keys"
Jul 28 21:26:33.251773 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.251756    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.254100 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.254084    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 21:26:33.254465 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.254448    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b439d9-serving-cert\""
Jul 28 21:26:33.254544 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.254478    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sz5mx\""
Jul 28 21:26:33.254609 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.254481    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:26:33.260096 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.259834    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:26:33.386290 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.386260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.386463 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.386365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kb4\" (UniqueName: \"kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.386463 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.386413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.487316 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.487277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.487530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.487335    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kb4\" (UniqueName: \"kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.487530 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:26:33.487442    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-serving-cert: secret "maas-api-e2e-shared-a-b439d9-serving-cert" not found
Jul 28 21:26:33.487530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.487501    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.487530 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:26:33.487529    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls podName:c9975420-58cd-4d15-9c4f-73da84e88eb8 nodeName:}" failed. No retries permitted until 2026-07-28 21:26:33.987503305 +0000 UTC m=+2488.386559952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls") pod "maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" (UID: "c9975420-58cd-4d15-9c4f-73da84e88eb8") : secret "maas-api-e2e-shared-a-b439d9-serving-cert" not found
Jul 28 21:26:33.488048 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.488031    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.501017 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.500966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kb4\" (UniqueName: \"kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.992891 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.992860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:33.995443 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:33.995421    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") pod \"maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:34.164153 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:34.164114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:34.312619 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:34.312590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:26:34.315066 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:26:34.315033    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9975420_58cd_4d15_9c4f_73da84e88eb8.slice/crio-849f14cbeb03fd17f36a51c5864a11e87769398945c3a6fad1379d91d1a0cd23 WatchSource:0}: Error finding container 849f14cbeb03fd17f36a51c5864a11e87769398945c3a6fad1379d91d1a0cd23: Status 404 returned error can't find the container with id 849f14cbeb03fd17f36a51c5864a11e87769398945c3a6fad1379d91d1a0cd23
Jul 28 21:26:35.019959 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:35.019921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" event={"ID":"c9975420-58cd-4d15-9c4f-73da84e88eb8","Type":"ContainerStarted","Data":"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"}
Jul 28 21:26:35.020198 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:35.020014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" event={"ID":"c9975420-58cd-4d15-9c4f-73da84e88eb8","Type":"ContainerStarted","Data":"849f14cbeb03fd17f36a51c5864a11e87769398945c3a6fad1379d91d1a0cd23"}
Jul 28 21:26:35.020198 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:35.020186    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:26:35.039843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:35.039792    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" podStartSLOduration=1.5287304019999999 podStartE2EDuration="2.039776383s" totalImagesPullingTime="511.045981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:26:35.037513807 +0000 UTC m=+2489.436570474" watchObservedRunningTime="2026-07-28 21:26:35.039776383 +0000 UTC m=+2489.438833058"
Jul 28 21:26:37.844857 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:37.844822    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:26:41.030799 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:26:41.030768    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:27:02.272889 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.272853    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:27:02.276838 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.276815    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.279006 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.278984    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-7076b1-serving-cert\""
Jul 28 21:27:02.290926 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.290897    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:27:02.339105 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.339065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.339348 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.339110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpg79\" (UniqueName: \"kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.339348 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.339235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.439927 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.439893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.440119 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.439995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.440119 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.440023    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpg79\" (UniqueName: \"kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.440651 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.440631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.442656 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.442634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.449368 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.449347    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpg79\" (UniqueName: \"kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79\") pod \"maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.586938 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.586853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:02.726865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:02.726835    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:27:02.728392 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:27:02.728369    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2596fa2_9fbc_4dfe_ba38_ae2800d1fb33.slice/crio-d94aab9d18cec2356268948fb4ba310e39d523c927cd00b71db625401a889781 WatchSource:0}: Error finding container d94aab9d18cec2356268948fb4ba310e39d523c927cd00b71db625401a889781: Status 404 returned error can't find the container with id d94aab9d18cec2356268948fb4ba310e39d523c927cd00b71db625401a889781
Jul 28 21:27:03.178841 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:03.178806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" event={"ID":"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33","Type":"ContainerStarted","Data":"d94aab9d18cec2356268948fb4ba310e39d523c927cd00b71db625401a889781"}
Jul 28 21:27:04.185848 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:04.185811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" event={"ID":"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33","Type":"ContainerStarted","Data":"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"}
Jul 28 21:27:04.186317 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:04.185882    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:04.203098 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:04.203034    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" podStartSLOduration=1.810489119 podStartE2EDuration="2.203014081s" totalImagesPullingTime="392.524962ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:27:04.202798362 +0000 UTC m=+2518.601855027" watchObservedRunningTime="2026-07-28 21:27:04.203014081 +0000 UTC m=+2518.602070747"
Jul 28 21:27:10.046979 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:10.046938    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:27:10.197645 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:10.197619    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:27:43.844481 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:27:43.844441    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:28:26.540381 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:28:26.540344    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:28:59.940848 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:28:59.940813    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:29:23.545553 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:29:23.545520    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:29:41.341675 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:29:41.341640    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:30:00.143327 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.143247    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"]
Jul 28 21:30:00.146996 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.146973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"
Jul 28 21:30:00.149615 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.149596    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:30:00.157929 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.157910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"
Jul 28 21:30:00.158242 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.158222    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"]
Jul 28 21:30:00.291272 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:00.291246    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"]
Jul 28 21:30:00.293890 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:30:00.293867    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff5a337_8a17_4132_881e_317babd7a653.slice/crio-c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562 WatchSource:0}: Error finding container c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562: Status 404 returned error can't find the container with id c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562
Jul 28 21:30:01.154354 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:01.154320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz" event={"ID":"5ff5a337-8a17-4132-881e-317babd7a653","Type":"ContainerStarted","Data":"f6cefa455877eb53e787faace0640b910ef3e5e2facb200f6c2315b30f8fb3e3"}
Jul 28 21:30:01.154728 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:01.154369    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz" event={"ID":"5ff5a337-8a17-4132-881e-317babd7a653","Type":"ContainerStarted","Data":"c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562"}
Jul 28 21:30:01.171271 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:01.171231    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz" podStartSLOduration=1.171217416 podStartE2EDuration="1.171217416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:30:01.169252949 +0000 UTC m=+2695.568309614" watchObservedRunningTime="2026-07-28 21:30:01.171217416 +0000 UTC m=+2695.570274081"
Jul 28 21:30:02.160670 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:02.160633    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ff5a337-8a17-4132-881e-317babd7a653" containerID="f6cefa455877eb53e787faace0640b910ef3e5e2facb200f6c2315b30f8fb3e3" exitCode=0
Jul 28 21:30:02.161048 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:02.160677    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz" event={"ID":"5ff5a337-8a17-4132-881e-317babd7a653","Type":"ContainerDied","Data":"f6cefa455877eb53e787faace0640b910ef3e5e2facb200f6c2315b30f8fb3e3"}
Jul 28 21:30:03.298233 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:03.298204    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"
Jul 28 21:30:04.173634 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:04.173595    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz" event={"ID":"5ff5a337-8a17-4132-881e-317babd7a653","Type":"ContainerDied","Data":"c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562"}
Jul 28 21:30:04.173634 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:04.173633    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754570-wg4pz"
Jul 28 21:30:04.173838 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:04.173650    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c380b953efeb8a0b75554802eaabc90dc21e7c1ddda44d5995717b7cec607562"
Jul 28 21:30:07.948202 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:07.948167    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:30:20.348337 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:20.348303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:30:40.246662 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:40.246623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:30:53.145972 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:30:53.145937    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:31:06.466581 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:31:06.466549    2567 scope.go:117] "RemoveContainer" containerID="9ddea44a76e5853a193aea3244659b1bf8df530919e90089bdb6ea392a73f4f1"
Jul 28 21:31:06.477861 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:31:06.477841    2567 scope.go:117] "RemoveContainer" containerID="ce62cdf94d21bba1a4d3f27e1f24f7efd445b8de640febefadaa719e08e20b8b"
Jul 28 21:31:14.372603 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:31:14.372560    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:31:27.149746 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:31:27.149670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:31:48.439694 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:31:48.439656    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:32:04.139292 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:32:04.139254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:32:06.502263 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:32:06.502233    2567 scope.go:117] "RemoveContainer" containerID="6e0597e6e35ac065a50689a4d530df4c06c795afab214b834446b20a8d48d9a7"
Jul 28 21:32:32.355545 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:32:32.355504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:32:50.361048 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:32:50.361014    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:32:59.651106 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:32:59.651027    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:33:06.524201 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:33:06.524166    2567 scope.go:117] "RemoveContainer" containerID="36b92f2f35d567a0e60f308a9b61e5208f2f6e3ee61348a580ef636a875b3ac0"
Jul 28 21:33:36.570445 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:33:36.570406    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:34:07.444554 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:34:07.444519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:34:26.245509 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:34:26.245472    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:34:44.546745 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:34:44.546662    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:34:54.047243 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:34:54.047209    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:35:31.038168 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.038123    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:35:31.038624 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.038501    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff5a337-8a17-4132-881e-317babd7a653" containerName="cleanup"
Jul 28 21:35:31.038624 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.038512    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff5a337-8a17-4132-881e-317babd7a653" containerName="cleanup"
Jul 28 21:35:31.038624 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.038589    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff5a337-8a17-4132-881e-317babd7a653" containerName="cleanup"
Jul 28 21:35:31.041566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.041544    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.043769 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.043752    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7076b1\"/\"default-dockercfg-84l9t\""
Jul 28 21:35:31.044358 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.044341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7076b1\"/\"openshift-service-ca.crt\""
Jul 28 21:35:31.044456 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.044394    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7076b1\"/\"kube-root-ca.crt\""
Jul 28 21:35:31.044456 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.044422    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7076b1\"/\"rate-test-model-7076b1-kserve-self-signed-certs\""
Jul 28 21:35:31.050967 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.050941    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:35:31.127183 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.127183 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127191    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.127392 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.127392 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.127392 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869ml\" (UniqueName: \"kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.127527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.127412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.227991 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.227957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869ml\" (UniqueName: \"kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228212 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228443 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228426    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228505 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228487    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.228557 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.228539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.230525 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.230504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.230760 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.230745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.236164 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.236124    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-869ml\" (UniqueName: \"kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml\") pod \"rate-test-model-7076b1-kserve-75677b9b59-xhj5h\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") " pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.352154 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.352050    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:31.486873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.486851    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:35:31.489066 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:35:31.488998    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87b511c0_43ac_4c86_80e9_3f50467ca242.slice/crio-0af55cfc3496510b368d4cd8ad4a49ebf2ec4c5eef9dd09a3ba8651d6eac0a5e WatchSource:0}: Error finding container 0af55cfc3496510b368d4cd8ad4a49ebf2ec4c5eef9dd09a3ba8651d6eac0a5e: Status 404 returned error can't find the container with id 0af55cfc3496510b368d4cd8ad4a49ebf2ec4c5eef9dd09a3ba8651d6eac0a5e
Jul 28 21:35:31.943462 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.943428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" event={"ID":"87b511c0-43ac-4c86-80e9-3f50467ca242","Type":"ContainerStarted","Data":"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"}
Jul 28 21:35:31.943462 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.943479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" event={"ID":"87b511c0-43ac-4c86-80e9-3f50467ca242","Type":"ContainerStarted","Data":"0af55cfc3496510b368d4cd8ad4a49ebf2ec4c5eef9dd09a3ba8651d6eac0a5e"}
Jul 28 21:35:31.960791 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:31.960745    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" podStartSLOduration=0.960730977 podStartE2EDuration="960.730977ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:35:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:35:31.958591551 +0000 UTC m=+3026.357648217" watchObservedRunningTime="2026-07-28 21:35:31.960730977 +0000 UTC m=+3026.359787641"
Jul 28 21:35:34.545832 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:34.545794    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:35:41.352304 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:41.352271    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:41.352755 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:41.352320    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:41.364684 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:41.364660    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:35:42.006069 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:35:42.006042    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:36:07.345345 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:07.345314    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:36:21.251114 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:21.251079    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:36:35.149080 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:35.149047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:36:40.329491 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:40.329461    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:36:41.339307 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.339274    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365412016"]
Jul 28 21:36:41.342756 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.342740    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:41.344710 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.344689    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5f98m\""
Jul 28 21:36:41.348067 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.348045    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365412016"]
Jul 28 21:36:41.441757 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.441714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssncs\" (UniqueName: \"kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs\") pod \"test-curl-7563-139971365412016\" (UID: \"98b65c69-2166-402c-ae77-9748d8fbaf5b\") " pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:41.542585 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.542560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssncs\" (UniqueName: \"kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs\") pod \"test-curl-7563-139971365412016\" (UID: \"98b65c69-2166-402c-ae77-9748d8fbaf5b\") " pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:41.550607 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.550575    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssncs\" (UniqueName: \"kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs\") pod \"test-curl-7563-139971365412016\" (UID: \"98b65c69-2166-402c-ae77-9748d8fbaf5b\") " pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:41.652885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.652804    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:41.783615 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.783595    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365412016"]
Jul 28 21:36:41.785475 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:41.785445    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b65c69_2166_402c_ae77_9748d8fbaf5b.slice/crio-479890d7535eca50908bfa59e543f5e002842de7a90bb81dd5e7f4c19b3530c6 WatchSource:0}: Error finding container 479890d7535eca50908bfa59e543f5e002842de7a90bb81dd5e7f4c19b3530c6: Status 404 returned error can't find the container with id 479890d7535eca50908bfa59e543f5e002842de7a90bb81dd5e7f4c19b3530c6
Jul 28 21:36:41.786920 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:41.786903    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:36:42.312625 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:42.312582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365412016" event={"ID":"98b65c69-2166-402c-ae77-9748d8fbaf5b","Type":"ContainerStarted","Data":"479890d7535eca50908bfa59e543f5e002842de7a90bb81dd5e7f4c19b3530c6"}
Jul 28 21:36:43.319381 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:43.319344    2567 generic.go:356] "Generic (PLEG): container finished" podID="98b65c69-2166-402c-ae77-9748d8fbaf5b" containerID="f50138754ffcadcb714317b8d1da37beaabb03fc2c0f2e09cac93c888f0e820c" exitCode=0
Jul 28 21:36:43.319781 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:43.319464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365412016" event={"ID":"98b65c69-2166-402c-ae77-9748d8fbaf5b","Type":"ContainerDied","Data":"f50138754ffcadcb714317b8d1da37beaabb03fc2c0f2e09cac93c888f0e820c"}
Jul 28 21:36:44.457998 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.457978    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:44.491660 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.491630    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365412016_98b65c69-2166-402c-ae77-9748d8fbaf5b/test-curl-7563-139971365412016/0.log"
Jul 28 21:36:44.521516 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.521489    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365412016"]
Jul 28 21:36:44.523571 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.523551    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365412016"]
Jul 28 21:36:44.571334 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.571306    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssncs\" (UniqueName: \"kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs\") pod \"98b65c69-2166-402c-ae77-9748d8fbaf5b\" (UID: \"98b65c69-2166-402c-ae77-9748d8fbaf5b\") "
Jul 28 21:36:44.573517 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.573493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs" (OuterVolumeSpecName: "kube-api-access-ssncs") pod "98b65c69-2166-402c-ae77-9748d8fbaf5b" (UID: "98b65c69-2166-402c-ae77-9748d8fbaf5b"). InnerVolumeSpecName "kube-api-access-ssncs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:44.653724 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.653693    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365412656"]
Jul 28 21:36:44.654072 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.654059    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" containerName="test-curl-7563-139971365412016"
Jul 28 21:36:44.654128 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.654073    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" containerName="test-curl-7563-139971365412016"
Jul 28 21:36:44.654202 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.654167    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" containerName="test-curl-7563-139971365412016"
Jul 28 21:36:44.657130 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.657114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:44.665724 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.665701    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365412656"]
Jul 28 21:36:44.672618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.672592    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssncs\" (UniqueName: \"kubernetes.io/projected/98b65c69-2166-402c-ae77-9748d8fbaf5b-kube-api-access-ssncs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:44.774058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.774011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmnrb\" (UniqueName: \"kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb\") pod \"test-curl-7563-139971365412656\" (UID: \"907e5c8f-a910-4ecf-b4d7-5443a680e8be\") " pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:44.874678 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.874573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmnrb\" (UniqueName: \"kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb\") pod \"test-curl-7563-139971365412656\" (UID: \"907e5c8f-a910-4ecf-b4d7-5443a680e8be\") " pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:44.884232 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.884209    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmnrb\" (UniqueName: \"kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb\") pod \"test-curl-7563-139971365412656\" (UID: \"907e5c8f-a910-4ecf-b4d7-5443a680e8be\") " pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:44.966513 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:44.966489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:45.096177 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:45.096154    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365412656"]
Jul 28 21:36:45.097805 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:45.097781    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod907e5c8f_a910_4ecf_b4d7_5443a680e8be.slice/crio-c61bc9c64890fe3a87b4279fef88ea0424e22fe0ee37166158c4a30aefecf47f WatchSource:0}: Error finding container c61bc9c64890fe3a87b4279fef88ea0424e22fe0ee37166158c4a30aefecf47f: Status 404 returned error can't find the container with id c61bc9c64890fe3a87b4279fef88ea0424e22fe0ee37166158c4a30aefecf47f
Jul 28 21:36:45.331489 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:45.331444    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="479890d7535eca50908bfa59e543f5e002842de7a90bb81dd5e7f4c19b3530c6"
Jul 28 21:36:45.331489 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:45.331459    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:45.332768 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:45.332736    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365412656" event={"ID":"907e5c8f-a910-4ecf-b4d7-5443a680e8be","Type":"ContainerStarted","Data":"c61bc9c64890fe3a87b4279fef88ea0424e22fe0ee37166158c4a30aefecf47f"}
Jul 28 21:36:45.341302 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:36:45.341275    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7563-139971365412016\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:46.075832 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:36:46.075795    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7563-139971365412016\" is forbidden: User \"system:node:ip-10-0-140-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-162.ec2.internal' and this object" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" pod="opendatahub/test-curl-7563-139971365412016"
Jul 28 21:36:46.077463 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:46.077444    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b65c69-2166-402c-ae77-9748d8fbaf5b" path="/var/lib/kubelet/pods/98b65c69-2166-402c-ae77-9748d8fbaf5b/volumes"
Jul 28 21:36:46.339567 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:46.339471    2567 generic.go:356] "Generic (PLEG): container finished" podID="907e5c8f-a910-4ecf-b4d7-5443a680e8be" containerID="80cdd5ba2a57a8ee9dced122597830aabc0908866b7c74d415f8bc3d5813770c" exitCode=0
Jul 28 21:36:46.339718 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:46.339562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365412656" event={"ID":"907e5c8f-a910-4ecf-b4d7-5443a680e8be","Type":"ContainerDied","Data":"80cdd5ba2a57a8ee9dced122597830aabc0908866b7c74d415f8bc3d5813770c"}
Jul 28 21:36:47.483124 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.483096    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:47.503071 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.503042    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365412656_907e5c8f-a910-4ecf-b4d7-5443a680e8be/test-curl-7563-139971365412656/0.log"
Jul 28 21:36:47.529470 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.529443    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365412656"]
Jul 28 21:36:47.533464 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.533438    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365412656"]
Jul 28 21:36:47.603945 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.603880    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmnrb\" (UniqueName: \"kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb\") pod \"907e5c8f-a910-4ecf-b4d7-5443a680e8be\" (UID: \"907e5c8f-a910-4ecf-b4d7-5443a680e8be\") "
Jul 28 21:36:47.606012 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.605992    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb" (OuterVolumeSpecName: "kube-api-access-bmnrb") pod "907e5c8f-a910-4ecf-b4d7-5443a680e8be" (UID: "907e5c8f-a910-4ecf-b4d7-5443a680e8be"). InnerVolumeSpecName "kube-api-access-bmnrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:47.701030 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.701002    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:47.701412 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.701398    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="907e5c8f-a910-4ecf-b4d7-5443a680e8be" containerName="test-curl-7563-139971365412656"
Jul 28 21:36:47.701465 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.701414    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="907e5c8f-a910-4ecf-b4d7-5443a680e8be" containerName="test-curl-7563-139971365412656"
Jul 28 21:36:47.701501 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.701489    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="907e5c8f-a910-4ecf-b4d7-5443a680e8be" containerName="test-curl-7563-139971365412656"
Jul 28 21:36:47.704374 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.704351    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:47.704671 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.704645    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmnrb\" (UniqueName: \"kubernetes.io/projected/907e5c8f-a910-4ecf-b4d7-5443a680e8be-kube-api-access-bmnrb\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:47.710180 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.710162    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:47.805788 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.805760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5bj7\" (UniqueName: \"kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7\") pod \"test-curl-7563-139971365419024\" (UID: \"750dde85-7312-4598-9ca0-d7c5bd69305d\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:47.906684 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.906594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5bj7\" (UniqueName: \"kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7\") pod \"test-curl-7563-139971365419024\" (UID: \"750dde85-7312-4598-9ca0-d7c5bd69305d\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:47.915010 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:47.914987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5bj7\" (UniqueName: \"kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7\") pod \"test-curl-7563-139971365419024\" (UID: \"750dde85-7312-4598-9ca0-d7c5bd69305d\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:48.015387 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.015335    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:48.079314 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.079285    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="907e5c8f-a910-4ecf-b4d7-5443a680e8be" path="/var/lib/kubelet/pods/907e5c8f-a910-4ecf-b4d7-5443a680e8be/volumes"
Jul 28 21:36:48.147589 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.147568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:48.149082 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:48.149049    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod750dde85_7312_4598_9ca0_d7c5bd69305d.slice/crio-851e697e8779bd481325021ae6166e0ce9964064dcbaba8d577d2957de0a0448 WatchSource:0}: Error finding container 851e697e8779bd481325021ae6166e0ce9964064dcbaba8d577d2957de0a0448: Status 404 returned error can't find the container with id 851e697e8779bd481325021ae6166e0ce9964064dcbaba8d577d2957de0a0448
Jul 28 21:36:48.351908 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.351877    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365412656"
Jul 28 21:36:48.351908 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.351891    2567 scope.go:117] "RemoveContainer" containerID="80cdd5ba2a57a8ee9dced122597830aabc0908866b7c74d415f8bc3d5813770c"
Jul 28 21:36:48.353351 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:48.353172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365419024" event={"ID":"750dde85-7312-4598-9ca0-d7c5bd69305d","Type":"ContainerStarted","Data":"851e697e8779bd481325021ae6166e0ce9964064dcbaba8d577d2957de0a0448"}
Jul 28 21:36:49.359942 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:49.359906    2567 generic.go:356] "Generic (PLEG): container finished" podID="750dde85-7312-4598-9ca0-d7c5bd69305d" containerID="e6b1a646402f85f5b11de85c3804352147ae033fbad4bf79c3210882c401f1c9" exitCode=0
Jul 28 21:36:49.360452 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:49.359996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365419024" event={"ID":"750dde85-7312-4598-9ca0-d7c5bd69305d","Type":"ContainerDied","Data":"e6b1a646402f85f5b11de85c3804352147ae033fbad4bf79c3210882c401f1c9"}
Jul 28 21:36:50.439237 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.439203    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:36:50.503635 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.503612    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:50.523038 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.523007    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365419024_750dde85-7312-4598-9ca0-d7c5bd69305d/test-curl-7563-139971365419024/0.log"
Jul 28 21:36:50.526319 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.526301    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5bj7\" (UniqueName: \"kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7\") pod \"750dde85-7312-4598-9ca0-d7c5bd69305d\" (UID: \"750dde85-7312-4598-9ca0-d7c5bd69305d\") "
Jul 28 21:36:50.528535 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.528512    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7" (OuterVolumeSpecName: "kube-api-access-r5bj7") pod "750dde85-7312-4598-9ca0-d7c5bd69305d" (UID: "750dde85-7312-4598-9ca0-d7c5bd69305d"). InnerVolumeSpecName "kube-api-access-r5bj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:50.548038 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.548012    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:50.551878 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.551858    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:50.627391 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.627340    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5bj7\" (UniqueName: \"kubernetes.io/projected/750dde85-7312-4598-9ca0-d7c5bd69305d-kube-api-access-r5bj7\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:50.676479 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.676460    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:50.676828 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.676816    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="750dde85-7312-4598-9ca0-d7c5bd69305d" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:50.676872 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.676830    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="750dde85-7312-4598-9ca0-d7c5bd69305d" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:50.676904 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.676891    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="750dde85-7312-4598-9ca0-d7c5bd69305d" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:50.679921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.679908    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:50.686702 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.686682    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:50.728042 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.728018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl5cw\" (UniqueName: \"kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw\") pod \"test-curl-7563-139971365419024\" (UID: \"2aca3b41-909e-4d3d-8880-dd7c801c42eb\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:50.828850 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.828824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl5cw\" (UniqueName: \"kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw\") pod \"test-curl-7563-139971365419024\" (UID: \"2aca3b41-909e-4d3d-8880-dd7c801c42eb\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:50.836903 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.836883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl5cw\" (UniqueName: \"kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw\") pod \"test-curl-7563-139971365419024\" (UID: \"2aca3b41-909e-4d3d-8880-dd7c801c42eb\") " pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:50.990225 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:50.990127    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:51.327524 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:51.327500    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:51.329265 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:51.329234    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2aca3b41_909e_4d3d_8880_dd7c801c42eb.slice/crio-d60a9282c3d19913ebae1f55f73ced704e5e3da69070a2b92e116da8f924cd9e WatchSource:0}: Error finding container d60a9282c3d19913ebae1f55f73ced704e5e3da69070a2b92e116da8f924cd9e: Status 404 returned error can't find the container with id d60a9282c3d19913ebae1f55f73ced704e5e3da69070a2b92e116da8f924cd9e
Jul 28 21:36:51.372825 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:51.372788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365419024" event={"ID":"2aca3b41-909e-4d3d-8880-dd7c801c42eb","Type":"ContainerStarted","Data":"d60a9282c3d19913ebae1f55f73ced704e5e3da69070a2b92e116da8f924cd9e"}
Jul 28 21:36:51.373991 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:51.373975    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="851e697e8779bd481325021ae6166e0ce9964064dcbaba8d577d2957de0a0448"
Jul 28 21:36:51.374090 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:51.374002    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:51.387031 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:51.387000    2567 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7563-139971365419024" oldPodUID="750dde85-7312-4598-9ca0-d7c5bd69305d" podUID="2aca3b41-909e-4d3d-8880-dd7c801c42eb"
Jul 28 21:36:52.076921 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:52.076890    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="750dde85-7312-4598-9ca0-d7c5bd69305d" path="/var/lib/kubelet/pods/750dde85-7312-4598-9ca0-d7c5bd69305d/volumes"
Jul 28 21:36:52.380721 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:52.380633    2567 generic.go:356] "Generic (PLEG): container finished" podID="2aca3b41-909e-4d3d-8880-dd7c801c42eb" containerID="19012c4243406f812e1da9941640797f6c31bd44b343e9eeb577b8ee4eb7f352" exitCode=0
Jul 28 21:36:52.380721 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:52.380673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365419024" event={"ID":"2aca3b41-909e-4d3d-8880-dd7c801c42eb","Type":"ContainerDied","Data":"19012c4243406f812e1da9941640797f6c31bd44b343e9eeb577b8ee4eb7f352"}
Jul 28 21:36:53.525885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.525864    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:53.545073 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.545035    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365419024_2aca3b41-909e-4d3d-8880-dd7c801c42eb/test-curl-7563-139971365419024/0.log"
Jul 28 21:36:53.551326 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.551304    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl5cw\" (UniqueName: \"kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw\") pod \"2aca3b41-909e-4d3d-8880-dd7c801c42eb\" (UID: \"2aca3b41-909e-4d3d-8880-dd7c801c42eb\") "
Jul 28 21:36:53.553411 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.553386    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw" (OuterVolumeSpecName: "kube-api-access-wl5cw") pod "2aca3b41-909e-4d3d-8880-dd7c801c42eb" (UID: "2aca3b41-909e-4d3d-8880-dd7c801c42eb"). InnerVolumeSpecName "kube-api-access-wl5cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:53.570426 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.570403    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:53.575884 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.575863    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365419024"]
Jul 28 21:36:53.652010 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.651957    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl5cw\" (UniqueName: \"kubernetes.io/projected/2aca3b41-909e-4d3d-8880-dd7c801c42eb-kube-api-access-wl5cw\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:53.699058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.699032    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:53.699434 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.699421    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aca3b41-909e-4d3d-8880-dd7c801c42eb" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:53.699495 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.699435    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aca3b41-909e-4d3d-8880-dd7c801c42eb" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:53.699541 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.699519    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aca3b41-909e-4d3d-8880-dd7c801c42eb" containerName="test-curl-7563-139971365419024"
Jul 28 21:36:53.702829 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.702814    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:53.708278 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.708255    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:53.752450 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.752421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqk66\" (UniqueName: \"kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66\") pod \"test-curl-7563-139971365420032\" (UID: \"6ced5546-d688-4c6c-aeeb-5187dd4c45e6\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:53.853079 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.853044    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqk66\" (UniqueName: \"kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66\") pod \"test-curl-7563-139971365420032\" (UID: \"6ced5546-d688-4c6c-aeeb-5187dd4c45e6\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:53.860677 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:53.860658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqk66\" (UniqueName: \"kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66\") pod \"test-curl-7563-139971365420032\" (UID: \"6ced5546-d688-4c6c-aeeb-5187dd4c45e6\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:54.012380 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.012353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:54.077682 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.077652    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aca3b41-909e-4d3d-8880-dd7c801c42eb" path="/var/lib/kubelet/pods/2aca3b41-909e-4d3d-8880-dd7c801c42eb/volumes"
Jul 28 21:36:54.146254 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.146232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:54.148022 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:54.147997    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ced5546_d688_4c6c_aeeb_5187dd4c45e6.slice/crio-ea575839c294efe4d0616a0bf54a047c96b17f93584efb7ce94a2a19e714bd3f WatchSource:0}: Error finding container ea575839c294efe4d0616a0bf54a047c96b17f93584efb7ce94a2a19e714bd3f: Status 404 returned error can't find the container with id ea575839c294efe4d0616a0bf54a047c96b17f93584efb7ce94a2a19e714bd3f
Jul 28 21:36:54.393368 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.393287    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365419024"
Jul 28 21:36:54.393368 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.393291    2567 scope.go:117] "RemoveContainer" containerID="19012c4243406f812e1da9941640797f6c31bd44b343e9eeb577b8ee4eb7f352"
Jul 28 21:36:54.394771 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:54.394515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365420032" event={"ID":"6ced5546-d688-4c6c-aeeb-5187dd4c45e6","Type":"ContainerStarted","Data":"ea575839c294efe4d0616a0bf54a047c96b17f93584efb7ce94a2a19e714bd3f"}
Jul 28 21:36:55.350291 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.350255    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" podUID="87b511c0-43ac-4c86-80e9-3f50467ca242" containerName="main" containerID="cri-o://a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8" gracePeriod=45
Jul 28 21:36:55.401676 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.401644    2567 generic.go:356] "Generic (PLEG): container finished" podID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" containerID="93785930b68cd994038af8641d36a8ba51ae8321b22f2206d5120fcf9ad1ab6a" exitCode=0
Jul 28 21:36:55.401799 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.401696    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365420032" event={"ID":"6ced5546-d688-4c6c-aeeb-5187dd4c45e6","Type":"ContainerDied","Data":"93785930b68cd994038af8641d36a8ba51ae8321b22f2206d5120fcf9ad1ab6a"}
Jul 28 21:36:55.595068 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.595049    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:36:55.667708 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667630    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667708 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667915 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667722    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-869ml\" (UniqueName: \"kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667915 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667756    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667915 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667796    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667915 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667848    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm\") pod \"87b511c0-43ac-4c86-80e9-3f50467ca242\" (UID: \"87b511c0-43ac-4c86-80e9-3f50467ca242\") "
Jul 28 21:36:55.667915 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667900    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:36:55.668191 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.667982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home" (OuterVolumeSpecName: "home") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:36:55.668191 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.668012    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache" (OuterVolumeSpecName: "model-cache") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:36:55.668293 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.668189    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-home\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:55.668293 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.668208    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-tmp-dir\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:55.668293 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.668224    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-model-cache\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:55.670092 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.670063    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml" (OuterVolumeSpecName: "kube-api-access-869ml") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "kube-api-access-869ml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:55.670092 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.670088    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm" (OuterVolumeSpecName: "dshm") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:36:55.670262 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.670150    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87b511c0-43ac-4c86-80e9-3f50467ca242" (UID: "87b511c0-43ac-4c86-80e9-3f50467ca242"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:36:55.768847 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.768816    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-869ml\" (UniqueName: \"kubernetes.io/projected/87b511c0-43ac-4c86-80e9-3f50467ca242-kube-api-access-869ml\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:55.768847 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.768842    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b511c0-43ac-4c86-80e9-3f50467ca242-tls-certs\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:55.769014 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:55.768854    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b511c0-43ac-4c86-80e9-3f50467ca242-dshm\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:56.409105 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.409022    2567 generic.go:356] "Generic (PLEG): container finished" podID="87b511c0-43ac-4c86-80e9-3f50467ca242" containerID="a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8" exitCode=0
Jul 28 21:36:56.409105 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.409065    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"
Jul 28 21:36:56.409658 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.409107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" event={"ID":"87b511c0-43ac-4c86-80e9-3f50467ca242","Type":"ContainerDied","Data":"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"}
Jul 28 21:36:56.409658 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.409159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h" event={"ID":"87b511c0-43ac-4c86-80e9-3f50467ca242","Type":"ContainerDied","Data":"0af55cfc3496510b368d4cd8ad4a49ebf2ec4c5eef9dd09a3ba8651d6eac0a5e"}
Jul 28 21:36:56.409658 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.409177    2567 scope.go:117] "RemoveContainer" containerID="a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"
Jul 28 21:36:56.418497 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.418481    2567 scope.go:117] "RemoveContainer" containerID="a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"
Jul 28 21:36:56.418759 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:36:56.418736    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8\": container with ID starting with a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8 not found: ID does not exist" containerID="a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"
Jul 28 21:36:56.418865 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.418768    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8"} err="failed to get container status \"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8\": rpc error: code = NotFound desc = could not find container \"a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8\": container with ID starting with a3c87808a93ada2046ac3d2e893228cd03499530667d7966701bfb14c9bd51d8 not found: ID does not exist"
Jul 28 21:36:56.424541 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.424516    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:36:56.430963 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.430943    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7076b1/rate-test-model-7076b1-kserve-75677b9b59-xhj5h"]
Jul 28 21:36:56.538572 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.538549    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:56.556855 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.556833    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365420032_6ced5546-d688-4c6c-aeeb-5187dd4c45e6/test-curl-7563-139971365420032/0.log"
Jul 28 21:36:56.576254 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.576231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqk66\" (UniqueName: \"kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66\") pod \"6ced5546-d688-4c6c-aeeb-5187dd4c45e6\" (UID: \"6ced5546-d688-4c6c-aeeb-5187dd4c45e6\") "
Jul 28 21:36:56.578561 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.578535    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66" (OuterVolumeSpecName: "kube-api-access-sqk66") pod "6ced5546-d688-4c6c-aeeb-5187dd4c45e6" (UID: "6ced5546-d688-4c6c-aeeb-5187dd4c45e6"). InnerVolumeSpecName "kube-api-access-sqk66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:56.581522 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.581484    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:56.585057 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.585031    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:56.677506 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.677415    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqk66\" (UniqueName: \"kubernetes.io/projected/6ced5546-d688-4c6c-aeeb-5187dd4c45e6-kube-api-access-sqk66\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:36:56.714173 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714128    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:56.714530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714517    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" containerName="test-curl-7563-139971365420032"
Jul 28 21:36:56.714530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714531    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" containerName="test-curl-7563-139971365420032"
Jul 28 21:36:56.714642 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714553    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87b511c0-43ac-4c86-80e9-3f50467ca242" containerName="main"
Jul 28 21:36:56.714642 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714558    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="87b511c0-43ac-4c86-80e9-3f50467ca242" containerName="main"
Jul 28 21:36:56.714642 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714614    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" containerName="test-curl-7563-139971365420032"
Jul 28 21:36:56.714642 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.714621    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="87b511c0-43ac-4c86-80e9-3f50467ca242" containerName="main"
Jul 28 21:36:56.717720 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.717703    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:56.721552 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.721519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:56.777942 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.777917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq7j2\" (UniqueName: \"kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2\") pod \"test-curl-7563-139971365420032\" (UID: \"e9c632e2-c43a-4a4f-8523-0f4012e41c5c\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:56.879402 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.879369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq7j2\" (UniqueName: \"kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2\") pod \"test-curl-7563-139971365420032\" (UID: \"e9c632e2-c43a-4a4f-8523-0f4012e41c5c\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:56.889250 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:56.889228    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq7j2\" (UniqueName: \"kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2\") pod \"test-curl-7563-139971365420032\" (UID: \"e9c632e2-c43a-4a4f-8523-0f4012e41c5c\") " pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:57.028028 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.028001    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:57.163220 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.163195    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:57.164860 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:36:57.164832    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9c632e2_c43a_4a4f_8523_0f4012e41c5c.slice/crio-8de6ccf24ed7b8dd233dd5c1ad609b5aa2668be35ec23285d38fd565e931d0d7 WatchSource:0}: Error finding container 8de6ccf24ed7b8dd233dd5c1ad609b5aa2668be35ec23285d38fd565e931d0d7: Status 404 returned error can't find the container with id 8de6ccf24ed7b8dd233dd5c1ad609b5aa2668be35ec23285d38fd565e931d0d7
Jul 28 21:36:57.415857 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.415769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365420032" event={"ID":"e9c632e2-c43a-4a4f-8523-0f4012e41c5c","Type":"ContainerStarted","Data":"8de6ccf24ed7b8dd233dd5c1ad609b5aa2668be35ec23285d38fd565e931d0d7"}
Jul 28 21:36:57.416878 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.416852    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:57.416995 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.416859    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea575839c294efe4d0616a0bf54a047c96b17f93584efb7ce94a2a19e714bd3f"
Jul 28 21:36:57.427648 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:57.427622    2567 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7563-139971365420032" oldPodUID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" podUID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c"
Jul 28 21:36:58.075729 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:58.075697    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ced5546-d688-4c6c-aeeb-5187dd4c45e6" path="/var/lib/kubelet/pods/6ced5546-d688-4c6c-aeeb-5187dd4c45e6/volumes"
Jul 28 21:36:58.076070 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:58.076057    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87b511c0-43ac-4c86-80e9-3f50467ca242" path="/var/lib/kubelet/pods/87b511c0-43ac-4c86-80e9-3f50467ca242/volumes"
Jul 28 21:36:58.423796 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:58.423703    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c" containerID="6acd7c209aeda3bf3e3e46f8dd1e58b1d3d75db5654b3000b645e9e6c1508efd" exitCode=0
Jul 28 21:36:58.423796 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:58.423765    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7563-139971365420032" event={"ID":"e9c632e2-c43a-4a4f-8523-0f4012e41c5c","Type":"ContainerDied","Data":"6acd7c209aeda3bf3e3e46f8dd1e58b1d3d75db5654b3000b645e9e6c1508efd"}
Jul 28 21:36:59.567353 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.567331    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:36:59.587725 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.587705    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7563-139971365420032_e9c632e2-c43a-4a4f-8523-0f4012e41c5c/test-curl-7563-139971365420032/0.log"
Jul 28 21:36:59.604892 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.604869    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq7j2\" (UniqueName: \"kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2\") pod \"e9c632e2-c43a-4a4f-8523-0f4012e41c5c\" (UID: \"e9c632e2-c43a-4a4f-8523-0f4012e41c5c\") "
Jul 28 21:36:59.606952 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.606926    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2" (OuterVolumeSpecName: "kube-api-access-wq7j2") pod "e9c632e2-c43a-4a4f-8523-0f4012e41c5c" (UID: "e9c632e2-c43a-4a4f-8523-0f4012e41c5c"). InnerVolumeSpecName "kube-api-access-wq7j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:36:59.610496 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.610475    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:59.614528 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.614506    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7563-139971365420032"]
Jul 28 21:36:59.706511 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:36:59.706436    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq7j2\" (UniqueName: \"kubernetes.io/projected/e9c632e2-c43a-4a4f-8523-0f4012e41c5c-kube-api-access-wq7j2\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:37:00.077492 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:00.077459    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c" path="/var/lib/kubelet/pods/e9c632e2-c43a-4a4f-8523-0f4012e41c5c/volumes"
Jul 28 21:37:00.436025 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:00.435951    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7563-139971365420032"
Jul 28 21:37:00.436025 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:00.435990    2567 scope.go:117] "RemoveContainer" containerID="6acd7c209aeda3bf3e3e46f8dd1e58b1d3d75db5654b3000b645e9e6c1508efd"
Jul 28 21:37:03.511774 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.511741    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:37:03.512168 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.512099    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c" containerName="test-curl-7563-139971365420032"
Jul 28 21:37:03.512168 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.512110    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c" containerName="test-curl-7563-139971365420032"
Jul 28 21:37:03.512256 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.512196    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c632e2-c43a-4a4f-8523-0f4012e41c5c" containerName="test-curl-7563-139971365420032"
Jul 28 21:37:03.516000 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.515980    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.518988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.518968    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7aec84-serving-cert\""
Jul 28 21:37:03.533846 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.533823    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:37:03.648705 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.648678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.648873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.648734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.648873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.648808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5dc7\" (UniqueName: \"kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.749560 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.749525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5dc7\" (UniqueName: \"kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.749560 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.749563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.749760 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.749616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.750374 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.750349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.752175 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.752160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.757297 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.757278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5dc7\" (UniqueName: \"kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7\") pod \"maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.826535 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.826454    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:03.968112 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:03.968086    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:37:03.970083 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:37:03.970057    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31c310d_f8bc_41e3_acaf_35c4cfcd5a81.slice/crio-8221d897b4c6086435d31a218b48171e88503b84b0d00ec4c9aab00f0dbd32c3 WatchSource:0}: Error finding container 8221d897b4c6086435d31a218b48171e88503b84b0d00ec4c9aab00f0dbd32c3: Status 404 returned error can't find the container with id 8221d897b4c6086435d31a218b48171e88503b84b0d00ec4c9aab00f0dbd32c3
Jul 28 21:37:04.460179 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:04.460118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" event={"ID":"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81","Type":"ContainerStarted","Data":"8221d897b4c6086435d31a218b48171e88503b84b0d00ec4c9aab00f0dbd32c3"}
Jul 28 21:37:05.468051 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:05.468011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" event={"ID":"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81","Type":"ContainerStarted","Data":"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"}
Jul 28 21:37:05.468486 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:05.468076    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:05.486944 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:05.486904    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" podStartSLOduration=1.8173176450000001 podStartE2EDuration="2.486887449s" totalImagesPullingTime="669.569804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:37:05.485456912 +0000 UTC m=+3119.884513576" watchObservedRunningTime="2026-07-28 21:37:05.486887449 +0000 UTC m=+3119.885944119"
Jul 28 21:37:11.478896 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:11.478859    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:37:11.944767 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:11.944735    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:37:33.488685 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.488605    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:37:33.492383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.492367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.494553 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.494528    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-35d85b-serving-cert\""
Jul 28 21:37:33.510018 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.509993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:37:33.601798 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.601768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tp75\" (UniqueName: \"kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.601928 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.601805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.601969 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.601928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.702364 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.702337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.702527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.702369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tp75\" (UniqueName: \"kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.702527 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.702394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.702527 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:37:33.702474    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-serving-cert: secret "maas-api-e2e-ipp-b-35d85b-serving-cert" not found
Jul 28 21:37:33.702639 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:37:33.702541    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls podName:95a616bb-b4f1-4451-9121-17eaf4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-28 21:37:34.202525572 +0000 UTC m=+3148.601582219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls") pod "maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" (UID: "95a616bb-b4f1-4451-9121-17eaf4cfb0d5") : secret "maas-api-e2e-ipp-b-35d85b-serving-cert" not found
Jul 28 21:37:33.702965 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.702950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:33.711302 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:33.711275    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tp75\" (UniqueName: \"kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:34.206234 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:34.206199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:34.208741 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:34.208714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:34.402506 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:34.402468    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:34.540193 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:34.540162    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:37:34.543252 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:37:34.543225    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a616bb_b4f1_4451_9121_17eaf4cfb0d5.slice/crio-88ce5252a9b7b487eb7c5eb5ae4228883321b94ccd65538b44f3ce85e331c56d WatchSource:0}: Error finding container 88ce5252a9b7b487eb7c5eb5ae4228883321b94ccd65538b44f3ce85e331c56d: Status 404 returned error can't find the container with id 88ce5252a9b7b487eb7c5eb5ae4228883321b94ccd65538b44f3ce85e331c56d
Jul 28 21:37:34.632419 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:34.632384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" event={"ID":"95a616bb-b4f1-4451-9121-17eaf4cfb0d5","Type":"ContainerStarted","Data":"88ce5252a9b7b487eb7c5eb5ae4228883321b94ccd65538b44f3ce85e331c56d"}
Jul 28 21:37:35.640304 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:35.640273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" event={"ID":"95a616bb-b4f1-4451-9121-17eaf4cfb0d5","Type":"ContainerStarted","Data":"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"}
Jul 28 21:37:35.640692 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:35.640343    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:35.655308 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:35.655262    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" podStartSLOduration=2.305694249 podStartE2EDuration="2.655247705s" totalImagesPullingTime="349.553456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:37:35.655239533 +0000 UTC m=+3150.054296200" watchObservedRunningTime="2026-07-28 21:37:35.655247705 +0000 UTC m=+3150.054304370"
Jul 28 21:37:41.651658 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:41.651627    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:37:42.141227 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:42.141197    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:37:52.086325 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.086296    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:37:52.089778 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.089761    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:37:52.092060 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.092037    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kmt5p\""
Jul 28 21:37:52.098954 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.098929    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:37:52.152994 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.152962    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cglp9\" (UniqueName: \"kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9\") pod \"kuadrant-operator-catalog-5mrfw\" (UID: \"835c0277-362c-4225-8387-382f6dbdea1f\") " pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:37:52.254429 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.254404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cglp9\" (UniqueName: \"kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9\") pod \"kuadrant-operator-catalog-5mrfw\" (UID: \"835c0277-362c-4225-8387-382f6dbdea1f\") " pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:37:52.262263 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.262244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cglp9\" (UniqueName: \"kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9\") pod \"kuadrant-operator-catalog-5mrfw\" (UID: \"835c0277-362c-4225-8387-382f6dbdea1f\") " pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:37:52.400224 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.400151    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:37:52.534333 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.534311    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:37:52.536166 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:37:52.536118    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835c0277_362c_4225_8387_382f6dbdea1f.slice/crio-cf4e09de24d0527249ea009d352b818adabc7f0ac2f685e5293ca0101872cb53 WatchSource:0}: Error finding container cf4e09de24d0527249ea009d352b818adabc7f0ac2f685e5293ca0101872cb53: Status 404 returned error can't find the container with id cf4e09de24d0527249ea009d352b818adabc7f0ac2f685e5293ca0101872cb53
Jul 28 21:37:52.734802 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:52.734725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" event={"ID":"835c0277-362c-4225-8387-382f6dbdea1f","Type":"ContainerStarted","Data":"cf4e09de24d0527249ea009d352b818adabc7f0ac2f685e5293ca0101872cb53"}
Jul 28 21:37:54.750713 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:54.750618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" event={"ID":"835c0277-362c-4225-8387-382f6dbdea1f","Type":"ContainerStarted","Data":"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"}
Jul 28 21:37:54.766755 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:37:54.766705    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" podStartSLOduration=0.857602779 podStartE2EDuration="2.766690651s" totalImagesPullingTime="1.909087872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:37:54.763489452 +0000 UTC m=+3169.162546118" watchObservedRunningTime="2026-07-28 21:37:54.766690651 +0000 UTC m=+3169.165747317"
Jul 28 21:38:02.400267 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:02.400215    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:02.400730 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:02.400302    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:02.422702 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:02.422676    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:02.816841 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:02.816812    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:02.852779 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:02.852749    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:38:03.461123 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:03.461091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:38:04.806361 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:04.806327    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" podUID="835c0277-362c-4225-8387-382f6dbdea1f" containerName="registry-server" containerID="cri-o://d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6" gracePeriod=2
Jul 28 21:38:05.055786 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.055763    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:05.173604 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.173526    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cglp9\" (UniqueName: \"kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9\") pod \"835c0277-362c-4225-8387-382f6dbdea1f\" (UID: \"835c0277-362c-4225-8387-382f6dbdea1f\") "
Jul 28 21:38:05.175685 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.175660    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9" (OuterVolumeSpecName: "kube-api-access-cglp9") pod "835c0277-362c-4225-8387-382f6dbdea1f" (UID: "835c0277-362c-4225-8387-382f6dbdea1f"). InnerVolumeSpecName "kube-api-access-cglp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:38:05.274197 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.274168    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cglp9\" (UniqueName: \"kubernetes.io/projected/835c0277-362c-4225-8387-382f6dbdea1f-kube-api-access-cglp9\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:38:05.814282 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.814249    2567 generic.go:356] "Generic (PLEG): container finished" podID="835c0277-362c-4225-8387-382f6dbdea1f" containerID="d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6" exitCode=0
Jul 28 21:38:05.814695 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.814288    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw"
Jul 28 21:38:05.814695 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.814334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" event={"ID":"835c0277-362c-4225-8387-382f6dbdea1f","Type":"ContainerDied","Data":"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"}
Jul 28 21:38:05.814695 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.814378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5mrfw" event={"ID":"835c0277-362c-4225-8387-382f6dbdea1f","Type":"ContainerDied","Data":"cf4e09de24d0527249ea009d352b818adabc7f0ac2f685e5293ca0101872cb53"}
Jul 28 21:38:05.814695 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.814402    2567 scope.go:117] "RemoveContainer" containerID="d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"
Jul 28 21:38:05.824190 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.824172    2567 scope.go:117] "RemoveContainer" containerID="d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"
Jul 28 21:38:05.824424 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:38:05.824406    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6\": container with ID starting with d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6 not found: ID does not exist" containerID="d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"
Jul 28 21:38:05.824477 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.824434    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6"} err="failed to get container status \"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6\": rpc error: code = NotFound desc = could not find container \"d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6\": container with ID starting with d4f688fbb2393c8978222bdd2dad14937824f22569dc63398007844e3cab6bb6 not found: ID does not exist"
Jul 28 21:38:05.834277 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.834244    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:38:05.836496 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:05.836474    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5mrfw"]
Jul 28 21:38:06.082715 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:06.082641    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="835c0277-362c-4225-8387-382f6dbdea1f" path="/var/lib/kubelet/pods/835c0277-362c-4225-8387-382f6dbdea1f/volumes"
Jul 28 21:38:34.148641 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:34.148604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:38:51.406866 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.406833    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:38:51.407448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.407423    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835c0277-362c-4225-8387-382f6dbdea1f" containerName="registry-server"
Jul 28 21:38:51.407448 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.407449    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="835c0277-362c-4225-8387-382f6dbdea1f" containerName="registry-server"
Jul 28 21:38:51.407652 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.407588    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="835c0277-362c-4225-8387-382f6dbdea1f" containerName="registry-server"
Jul 28 21:38:51.410736 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.410714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.413598 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.413578    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d4bcd2-serving-cert\""
Jul 28 21:38:51.427163 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.427119    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:38:51.477348 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.477309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t6s4\" (UniqueName: \"kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.477348 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.477349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.477558 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.477432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.577894 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.577862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.578051 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.577963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t6s4\" (UniqueName: \"kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.578051 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.577995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.578823 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.578766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.580612 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.580593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.586371 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.586348    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t6s4\" (UniqueName: \"kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4\") pod \"maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.721449 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.721358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:51.859362 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:51.859339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:38:51.861125 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:38:51.861090    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7a5985_aa92_42fb_800f_b0716ab81e4a.slice/crio-ce8aa0fd8ddf176200e85d5c46829f9500e59eb1f585011e07ef12d746673c8c WatchSource:0}: Error finding container ce8aa0fd8ddf176200e85d5c46829f9500e59eb1f585011e07ef12d746673c8c: Status 404 returned error can't find the container with id ce8aa0fd8ddf176200e85d5c46829f9500e59eb1f585011e07ef12d746673c8c
Jul 28 21:38:52.083294 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:52.083252    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" event={"ID":"4f7a5985-aa92-42fb-800f-b0716ab81e4a","Type":"ContainerStarted","Data":"ce8aa0fd8ddf176200e85d5c46829f9500e59eb1f585011e07ef12d746673c8c"}
Jul 28 21:38:53.087443 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:53.087405    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" event={"ID":"4f7a5985-aa92-42fb-800f-b0716ab81e4a","Type":"ContainerStarted","Data":"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"}
Jul 28 21:38:53.087884 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:53.087461    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:38:53.103401 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:53.103358    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" podStartSLOduration=1.751884928 podStartE2EDuration="2.103343806s" totalImagesPullingTime="351.458878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:38:53.101904804 +0000 UTC m=+3227.500961472" watchObservedRunningTime="2026-07-28 21:38:53.103343806 +0000 UTC m=+3227.502400470"
Jul 28 21:38:59.097631 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:38:59.097562    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:39:00.945078 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:00.945047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:39:16.609760 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.609726    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"]
Jul 28 21:39:16.613498 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.613478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:16.615547 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.615527    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:39:16.621276 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.621256    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"]
Jul 28 21:39:16.704042 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.704006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn\" (UID: \"91fb19f8-e1ac-40f8-bf22-982b8b950def\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:16.804765 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.804731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn\" (UID: \"91fb19f8-e1ac-40f8-bf22-982b8b950def\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:16.805427 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.805408    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn\" (UID: \"91fb19f8-e1ac-40f8-bf22-982b8b950def\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:16.925705 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:16.925628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:17.066109 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:17.066082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"]
Jul 28 21:39:17.067462 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:39:17.067427    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fb19f8_e1ac_40f8_bf22_982b8b950def.slice/crio-5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b WatchSource:0}: Error finding container 5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b: Status 404 returned error can't find the container with id 5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b
Jul 28 21:39:17.228500 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:17.228418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn" event={"ID":"91fb19f8-e1ac-40f8-bf22-982b8b950def","Type":"ContainerStarted","Data":"5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b"}
Jul 28 21:39:18.235388 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:18.235350    2567 generic.go:356] "Generic (PLEG): container finished" podID="91fb19f8-e1ac-40f8-bf22-982b8b950def" containerID="6e209588ff5130b4a23b56abf4b4eb7b91683398688d783f46def1af32c155f8" exitCode=0
Jul 28 21:39:18.235787 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:18.235438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn" event={"ID":"91fb19f8-e1ac-40f8-bf22-982b8b950def","Type":"ContainerDied","Data":"6e209588ff5130b4a23b56abf4b4eb7b91683398688d783f46def1af32c155f8"}
Jul 28 21:39:19.375210 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:19.375186    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:19.528320 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:19.528291    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca\") pod \"91fb19f8-e1ac-40f8-bf22-982b8b950def\" (UID: \"91fb19f8-e1ac-40f8-bf22-982b8b950def\") "
Jul 28 21:39:19.528622 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:19.528602    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91fb19f8-e1ac-40f8-bf22-982b8b950def" (UID: "91fb19f8-e1ac-40f8-bf22-982b8b950def"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:39:19.629343 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:19.629313    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fb19f8-e1ac-40f8-bf22-982b8b950def-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:20.247318 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:20.247289    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"
Jul 28 21:39:20.247488 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:20.247328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn" event={"ID":"91fb19f8-e1ac-40f8-bf22-982b8b950def","Type":"ContainerDied","Data":"5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b"}
Jul 28 21:39:20.247488 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:20.247378    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a3da36c24fd3e733ebf7246a47b3ca4167f92b26b16417c5463ae3966884e9b"
Jul 28 21:39:21.694125 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:21.694093    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:39:21.694732 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:21.694348    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" podUID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" containerName="maas-api" containerID="cri-o://7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d" gracePeriod=30
Jul 28 21:39:21.881428 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:21.881399    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"]
Jul 28 21:39:21.885114 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:21.885085    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d4bcd2-45fzn"]
Jul 28 21:39:21.945369 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:21.945321    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:39:22.052242 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.052216    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls\") pod \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") "
Jul 28 21:39:22.052407 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.052279    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca\") pod \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") "
Jul 28 21:39:22.052407 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.052308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t6s4\" (UniqueName: \"kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4\") pod \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\" (UID: \"4f7a5985-aa92-42fb-800f-b0716ab81e4a\") "
Jul 28 21:39:22.052632 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.052601    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f7a5985-aa92-42fb-800f-b0716ab81e4a" (UID: "4f7a5985-aa92-42fb-800f-b0716ab81e4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:39:22.054378 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.054354    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f7a5985-aa92-42fb-800f-b0716ab81e4a" (UID: "4f7a5985-aa92-42fb-800f-b0716ab81e4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:39:22.054545 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.054521    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4" (OuterVolumeSpecName: "kube-api-access-4t6s4") pod "4f7a5985-aa92-42fb-800f-b0716ab81e4a" (UID: "4f7a5985-aa92-42fb-800f-b0716ab81e4a"). InnerVolumeSpecName "kube-api-access-4t6s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:39:22.077342 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.077306    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91fb19f8-e1ac-40f8-bf22-982b8b950def" path="/var/lib/kubelet/pods/91fb19f8-e1ac-40f8-bf22-982b8b950def/volumes"
Jul 28 21:39:22.153691 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.153669    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f7a5985-aa92-42fb-800f-b0716ab81e4a-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:22.153691 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.153690    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7a5985-aa92-42fb-800f-b0716ab81e4a-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:22.153850 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.153700    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t6s4\" (UniqueName: \"kubernetes.io/projected/4f7a5985-aa92-42fb-800f-b0716ab81e4a-kube-api-access-4t6s4\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:22.259868 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.259840    2567 generic.go:356] "Generic (PLEG): container finished" podID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" containerID="7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d" exitCode=0
Jul 28 21:39:22.259988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.259885    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"
Jul 28 21:39:22.259988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.259932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" event={"ID":"4f7a5985-aa92-42fb-800f-b0716ab81e4a","Type":"ContainerDied","Data":"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"}
Jul 28 21:39:22.259988 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.259983    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22" event={"ID":"4f7a5985-aa92-42fb-800f-b0716ab81e4a","Type":"ContainerDied","Data":"ce8aa0fd8ddf176200e85d5c46829f9500e59eb1f585011e07ef12d746673c8c"}
Jul 28 21:39:22.260090 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.260012    2567 scope.go:117] "RemoveContainer" containerID="7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"
Jul 28 21:39:22.269374 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.269358    2567 scope.go:117] "RemoveContainer" containerID="7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"
Jul 28 21:39:22.269627 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:39:22.269611    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d\": container with ID starting with 7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d not found: ID does not exist" containerID="7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"
Jul 28 21:39:22.269672 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.269642    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d"} err="failed to get container status \"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d\": rpc error: code = NotFound desc = could not find container \"7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d\": container with ID starting with 7ac3ae6c6eaeaab5033585cac68789b8db83f559b8a0e9cf281490d85bdfc94d not found: ID does not exist"
Jul 28 21:39:22.276585 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.276559    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:39:22.280097 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:22.280077    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d4bcd2-5fdccbb576-8df22"]
Jul 28 21:39:24.076706 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:24.076677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" path="/var/lib/kubelet/pods/4f7a5985-aa92-42fb-800f-b0716ab81e4a/volumes"
Jul 28 21:39:26.250346 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:26.250304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:39:32.987551 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.987503    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"]
Jul 28 21:39:32.988007 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.987923    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" containerName="maas-api"
Jul 28 21:39:32.988007 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.987941    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" containerName="maas-api"
Jul 28 21:39:32.988007 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.987977    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91fb19f8-e1ac-40f8-bf22-982b8b950def" containerName="revoke-keys"
Jul 28 21:39:32.988007 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.987987    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91fb19f8-e1ac-40f8-bf22-982b8b950def" containerName="revoke-keys"
Jul 28 21:39:32.988174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.988094    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91fb19f8-e1ac-40f8-bf22-982b8b950def" containerName="revoke-keys"
Jul 28 21:39:32.988174 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.988109    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f7a5985-aa92-42fb-800f-b0716ab81e4a" containerName="maas-api"
Jul 28 21:39:32.992722 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.992704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:32.995259 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.995233    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:39:32.997266 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:32.997238    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"]
Jul 28 21:39:33.148280 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:33.148246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw\" (UID: \"2747705c-69fc-48a2-929c-dcd667755ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:33.249462 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:33.249387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw\" (UID: \"2747705c-69fc-48a2-929c-dcd667755ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:33.249979 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:33.249960    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw\" (UID: \"2747705c-69fc-48a2-929c-dcd667755ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:33.305402 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:33.305360    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:33.441271 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:33.441247    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"]
Jul 28 21:39:33.443165 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:39:33.443115    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2747705c_69fc_48a2_929c_dcd667755ccc.slice/crio-fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc WatchSource:0}: Error finding container fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc: Status 404 returned error can't find the container with id fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc
Jul 28 21:39:34.334675 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:34.334636    2567 generic.go:356] "Generic (PLEG): container finished" podID="2747705c-69fc-48a2-929c-dcd667755ccc" containerID="c88a9e6f4cc5793a9a81fb50671b6bc9e5d233c3f911a6f4af54019b185a39b1" exitCode=0
Jul 28 21:39:34.335077 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:34.334718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw" event={"ID":"2747705c-69fc-48a2-929c-dcd667755ccc","Type":"ContainerDied","Data":"c88a9e6f4cc5793a9a81fb50671b6bc9e5d233c3f911a6f4af54019b185a39b1"}
Jul 28 21:39:34.335077 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:34.334779    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw" event={"ID":"2747705c-69fc-48a2-929c-dcd667755ccc","Type":"ContainerStarted","Data":"fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc"}
Jul 28 21:39:35.476832 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:35.476809    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:35.669885 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:35.669805    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca\") pod \"2747705c-69fc-48a2-929c-dcd667755ccc\" (UID: \"2747705c-69fc-48a2-929c-dcd667755ccc\") "
Jul 28 21:39:35.670131 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:35.670109    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2747705c-69fc-48a2-929c-dcd667755ccc" (UID: "2747705c-69fc-48a2-929c-dcd667755ccc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:39:35.771473 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:35.771439    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2747705c-69fc-48a2-929c-dcd667755ccc-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:36.346057 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:36.345977    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"
Jul 28 21:39:36.346057 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:36.346009    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw" event={"ID":"2747705c-69fc-48a2-929c-dcd667755ccc","Type":"ContainerDied","Data":"fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc"}
Jul 28 21:39:36.346279 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:36.346085    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa3bb0296600293a898fb0398e47d8fbd34e7fb2801b11fd2f0e9eabbee7f3cc"
Jul 28 21:39:44.545281 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:44.545243    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:39:44.681816 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:44.681786    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:39:44.682041 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:44.682017    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" podUID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" containerName="maas-api" containerID="cri-o://bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b" gracePeriod=30
Jul 28 21:39:45.040082 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.040035    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"]
Jul 28 21:39:45.044623 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.044592    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7aec84-lpcqw"]
Jul 28 21:39:45.932888 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.932866    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:39:45.953673 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.953640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca\") pod \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") "
Jul 28 21:39:45.953780 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.953729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5dc7\" (UniqueName: \"kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7\") pod \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") "
Jul 28 21:39:45.953780 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.953773    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls\") pod \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\" (UID: \"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81\") "
Jul 28 21:39:45.954033 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.954006    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" (UID: "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:39:45.954127 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.954099    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:45.956130 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.956103    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" (UID: "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:39:45.956370 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:45.956347    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7" (OuterVolumeSpecName: "kube-api-access-b5dc7") pod "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" (UID: "c31c310d-f8bc-41e3-acaf-35c4cfcd5a81"). InnerVolumeSpecName "kube-api-access-b5dc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:39:46.055127 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.055058    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5dc7\" (UniqueName: \"kubernetes.io/projected/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-kube-api-access-b5dc7\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:46.055127 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.055084    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:39:46.075784 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.075754    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2747705c-69fc-48a2-929c-dcd667755ccc" path="/var/lib/kubelet/pods/2747705c-69fc-48a2-929c-dcd667755ccc/volumes"
Jul 28 21:39:46.403849 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.403758    2567 generic.go:356] "Generic (PLEG): container finished" podID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" containerID="bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b" exitCode=0
Jul 28 21:39:46.403849 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.403801    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"
Jul 28 21:39:46.403849 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.403843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" event={"ID":"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81","Type":"ContainerDied","Data":"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"}
Jul 28 21:39:46.404122 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.403891    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm" event={"ID":"c31c310d-f8bc-41e3-acaf-35c4cfcd5a81","Type":"ContainerDied","Data":"8221d897b4c6086435d31a218b48171e88503b84b0d00ec4c9aab00f0dbd32c3"}
Jul 28 21:39:46.404122 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.403925    2567 scope.go:117] "RemoveContainer" containerID="bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"
Jul 28 21:39:46.413077 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.413060    2567 scope.go:117] "RemoveContainer" containerID="bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"
Jul 28 21:39:46.413357 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:39:46.413338    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b\": container with ID starting with bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b not found: ID does not exist" containerID="bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"
Jul 28 21:39:46.413449 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.413375    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b"} err="failed to get container status \"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b\": rpc error: code = NotFound desc = could not find container \"bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b\": container with ID starting with bb04ab08fcfe76826f8dca51a08463b0c4b04ec556dda50e2d1d9a46a946021b not found: ID does not exist"
Jul 28 21:39:46.419129 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.419108    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:39:46.422830 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:46.422810    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7aec84-648cb7857b-n4nbm"]
Jul 28 21:39:48.081538 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:48.081506    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" path="/var/lib/kubelet/pods/c31c310d-f8bc-41e3-acaf-35c4cfcd5a81/volumes"
Jul 28 21:39:55.639401 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:39:55.639359    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:40:50.944943 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.944904    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"]
Jul 28 21:40:50.945383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945335    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" containerName="maas-api"
Jul 28 21:40:50.945383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" containerName="maas-api"
Jul 28 21:40:50.945383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945361    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2747705c-69fc-48a2-929c-dcd667755ccc" containerName="revoke-keys"
Jul 28 21:40:50.945383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2747705c-69fc-48a2-929c-dcd667755ccc" containerName="revoke-keys"
Jul 28 21:40:50.945517 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945430    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2747705c-69fc-48a2-929c-dcd667755ccc" containerName="revoke-keys"
Jul 28 21:40:50.945517 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.945441    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31c310d-f8bc-41e3-acaf-35c4cfcd5a81" containerName="maas-api"
Jul 28 21:40:50.948574 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.948556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:50.950590 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.950568    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:40:50.954216 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:50.954190    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"]
Jul 28 21:40:51.050592 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.050564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7\" (UID: \"85620adc-1606-4f79-b6fe-3db9e4fe1781\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:51.151031 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.150989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7\" (UID: \"85620adc-1606-4f79-b6fe-3db9e4fe1781\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:51.151690 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.151667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7\" (UID: \"85620adc-1606-4f79-b6fe-3db9e4fe1781\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:51.260677 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.260649    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:51.397710 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.397686    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"]
Jul 28 21:40:51.399865 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:40:51.399837    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85620adc_1606_4f79_b6fe_3db9e4fe1781.slice/crio-471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444 WatchSource:0}: Error finding container 471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444: Status 404 returned error can't find the container with id 471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444
Jul 28 21:40:51.766220 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.766187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7" event={"ID":"85620adc-1606-4f79-b6fe-3db9e4fe1781","Type":"ContainerStarted","Data":"8c2f8862a2e408c70a2f6105c9abfb7460688ad61d2ac51e1555c5d1713247a0"}
Jul 28 21:40:51.766341 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.766242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7" event={"ID":"85620adc-1606-4f79-b6fe-3db9e4fe1781","Type":"ContainerStarted","Data":"471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444"}
Jul 28 21:40:51.781814 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:51.781774    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7" podStartSLOduration=1.781759662 podStartE2EDuration="1.781759662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:40:51.780951071 +0000 UTC m=+3346.180007737" watchObservedRunningTime="2026-07-28 21:40:51.781759662 +0000 UTC m=+3346.180816327"
Jul 28 21:40:52.773235 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:52.773200    2567 generic.go:356] "Generic (PLEG): container finished" podID="85620adc-1606-4f79-b6fe-3db9e4fe1781" containerID="8c2f8862a2e408c70a2f6105c9abfb7460688ad61d2ac51e1555c5d1713247a0" exitCode=0
Jul 28 21:40:52.773630 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:52.773233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7" event={"ID":"85620adc-1606-4f79-b6fe-3db9e4fe1781","Type":"ContainerDied","Data":"8c2f8862a2e408c70a2f6105c9abfb7460688ad61d2ac51e1555c5d1713247a0"}
Jul 28 21:40:53.912830 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:53.912804    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:54.078508 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.078439    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca\") pod \"85620adc-1606-4f79-b6fe-3db9e4fe1781\" (UID: \"85620adc-1606-4f79-b6fe-3db9e4fe1781\") "
Jul 28 21:40:54.078754 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.078735    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85620adc-1606-4f79-b6fe-3db9e4fe1781" (UID: "85620adc-1606-4f79-b6fe-3db9e4fe1781"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:40:54.179567 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.179537    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85620adc-1606-4f79-b6fe-3db9e4fe1781-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:40:54.791239 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.791198    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7" event={"ID":"85620adc-1606-4f79-b6fe-3db9e4fe1781","Type":"ContainerDied","Data":"471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444"}
Jul 28 21:40:54.791239 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.791231    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"
Jul 28 21:40:54.791461 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:54.791286    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="471b7fd9e62246b32e65ef1936e43b07c4515fd26c2494aea851fd9eea28f444"
Jul 28 21:40:56.951473 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:56.951437    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:40:56.951959 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:56.951697    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" podUID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" containerName="maas-api" containerID="cri-o://3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b" gracePeriod=30
Jul 28 21:40:57.228130 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.228109    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:40:57.240545 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.240518    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"]
Jul 28 21:40:57.243390 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.243369    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-35d85b-677z7"]
Jul 28 21:40:57.309434 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.309404    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca\") pod \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") "
Jul 28 21:40:57.309578 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.309464    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") pod \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") "
Jul 28 21:40:57.309578 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.309501    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tp75\" (UniqueName: \"kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75\") pod \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\" (UID: \"95a616bb-b4f1-4451-9121-17eaf4cfb0d5\") "
Jul 28 21:40:57.309821 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.309790    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95a616bb-b4f1-4451-9121-17eaf4cfb0d5" (UID: "95a616bb-b4f1-4451-9121-17eaf4cfb0d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:40:57.311722 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.311700    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95a616bb-b4f1-4451-9121-17eaf4cfb0d5" (UID: "95a616bb-b4f1-4451-9121-17eaf4cfb0d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:40:57.311831 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.311746    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75" (OuterVolumeSpecName: "kube-api-access-5tp75") pod "95a616bb-b4f1-4451-9121-17eaf4cfb0d5" (UID: "95a616bb-b4f1-4451-9121-17eaf4cfb0d5"). InnerVolumeSpecName "kube-api-access-5tp75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:40:57.410792 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.410761    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:40:57.410792 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.410788    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tp75\" (UniqueName: \"kubernetes.io/projected/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-kube-api-access-5tp75\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:40:57.410792 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.410797    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a616bb-b4f1-4451-9121-17eaf4cfb0d5-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:40:57.809512 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.809473    2567 generic.go:356] "Generic (PLEG): container finished" podID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" containerID="3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b" exitCode=0
Jul 28 21:40:57.809719 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.809522    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"
Jul 28 21:40:57.809719 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.809523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" event={"ID":"95a616bb-b4f1-4451-9121-17eaf4cfb0d5","Type":"ContainerDied","Data":"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"}
Jul 28 21:40:57.809719 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.809582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm" event={"ID":"95a616bb-b4f1-4451-9121-17eaf4cfb0d5","Type":"ContainerDied","Data":"88ce5252a9b7b487eb7c5eb5ae4228883321b94ccd65538b44f3ce85e331c56d"}
Jul 28 21:40:57.809719 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.809604    2567 scope.go:117] "RemoveContainer" containerID="3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"
Jul 28 21:40:57.820159 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.820126    2567 scope.go:117] "RemoveContainer" containerID="3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"
Jul 28 21:40:57.820415 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:40:57.820395    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b\": container with ID starting with 3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b not found: ID does not exist" containerID="3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"
Jul 28 21:40:57.820465 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.820424    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b"} err="failed to get container status \"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b\": rpc error: code = NotFound desc = could not find container \"3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b\": container with ID starting with 3fd5259e6892edf47d177ea85c07c438db10d04e7b98ab69788246db09d5459b not found: ID does not exist"
Jul 28 21:40:57.830873 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.830850    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:40:57.834742 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:57.834720    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-35d85b-7f667b6cf6-jnfbm"]
Jul 28 21:40:58.077191 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:58.077098    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85620adc-1606-4f79-b6fe-3db9e4fe1781" path="/var/lib/kubelet/pods/85620adc-1606-4f79-b6fe-3db9e4fe1781/volumes"
Jul 28 21:40:58.077541 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:40:58.077513    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" path="/var/lib/kubelet/pods/95a616bb-b4f1-4451-9121-17eaf4cfb0d5/volumes"
Jul 28 21:41:00.439705 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:00.439670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:41:12.412587 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.412555    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:41:12.412957 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.412752    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" podUID="c9975420-58cd-4d15-9c4f-73da84e88eb8" containerName="maas-api" containerID="cri-o://98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f" gracePeriod=30
Jul 28 21:41:12.652204 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.652173    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:41:12.739917 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.739831    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5kb4\" (UniqueName: \"kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4\") pod \"c9975420-58cd-4d15-9c4f-73da84e88eb8\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") "
Jul 28 21:41:12.739917 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.739895    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") pod \"c9975420-58cd-4d15-9c4f-73da84e88eb8\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") "
Jul 28 21:41:12.739917 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.739914    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca\") pod \"c9975420-58cd-4d15-9c4f-73da84e88eb8\" (UID: \"c9975420-58cd-4d15-9c4f-73da84e88eb8\") "
Jul 28 21:41:12.740352 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.740329    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9975420-58cd-4d15-9c4f-73da84e88eb8" (UID: "c9975420-58cd-4d15-9c4f-73da84e88eb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:41:12.742179 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.742148    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9975420-58cd-4d15-9c4f-73da84e88eb8" (UID: "c9975420-58cd-4d15-9c4f-73da84e88eb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:41:12.742285 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.742218    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4" (OuterVolumeSpecName: "kube-api-access-v5kb4") pod "c9975420-58cd-4d15-9c4f-73da84e88eb8" (UID: "c9975420-58cd-4d15-9c4f-73da84e88eb8"). InnerVolumeSpecName "kube-api-access-v5kb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:41:12.840929 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.840896    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5kb4\" (UniqueName: \"kubernetes.io/projected/c9975420-58cd-4d15-9c4f-73da84e88eb8-kube-api-access-v5kb4\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:12.840929 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.840925    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9975420-58cd-4d15-9c4f-73da84e88eb8-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:12.840929 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.840935    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9975420-58cd-4d15-9c4f-73da84e88eb8-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:12.898720 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.898687    2567 generic.go:356] "Generic (PLEG): container finished" podID="c9975420-58cd-4d15-9c4f-73da84e88eb8" containerID="98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f" exitCode=0
Jul 28 21:41:12.898879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.898732    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"
Jul 28 21:41:12.898879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.898753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" event={"ID":"c9975420-58cd-4d15-9c4f-73da84e88eb8","Type":"ContainerDied","Data":"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"}
Jul 28 21:41:12.898879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.898803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g" event={"ID":"c9975420-58cd-4d15-9c4f-73da84e88eb8","Type":"ContainerDied","Data":"849f14cbeb03fd17f36a51c5864a11e87769398945c3a6fad1379d91d1a0cd23"}
Jul 28 21:41:12.898879 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.898827    2567 scope.go:117] "RemoveContainer" containerID="98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"
Jul 28 21:41:12.908492 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.908476    2567 scope.go:117] "RemoveContainer" containerID="98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"
Jul 28 21:41:12.908745 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:41:12.908726    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f\": container with ID starting with 98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f not found: ID does not exist" containerID="98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"
Jul 28 21:41:12.908812 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.908774    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f"} err="failed to get container status \"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f\": rpc error: code = NotFound desc = could not find container \"98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f\": container with ID starting with 98b9f9e84a08e3fcde1402b452601063954ec839ace95494024c4f58b344ba4f not found: ID does not exist"
Jul 28 21:41:12.919704 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.919680    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:41:12.923328 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:12.923307    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b439d9-5dbddcd999-zbg2g"]
Jul 28 21:41:14.076524 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:14.076490    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9975420-58cd-4d15-9c4f-73da84e88eb8" path="/var/lib/kubelet/pods/c9975420-58cd-4d15-9c4f-73da84e88eb8/volumes"
Jul 28 21:41:15.501632 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:15.501597    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:41:20.670287 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670252    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"]
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670608    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9975420-58cd-4d15-9c4f-73da84e88eb8" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670621    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9975420-58cd-4d15-9c4f-73da84e88eb8" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670629    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670634    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670685    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85620adc-1606-4f79-b6fe-3db9e4fe1781" containerName="revoke-keys"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670692    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="85620adc-1606-4f79-b6fe-3db9e4fe1781" containerName="revoke-keys"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670750    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a616bb-b4f1-4451-9121-17eaf4cfb0d5" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670758    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9975420-58cd-4d15-9c4f-73da84e88eb8" containerName="maas-api"
Jul 28 21:41:20.670770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.670769    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="85620adc-1606-4f79-b6fe-3db9e4fe1781" containerName="revoke-keys"
Jul 28 21:41:20.673572 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.673556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:20.675744 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.675721    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wt6gl\""
Jul 28 21:41:20.681300 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.681276    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"]
Jul 28 21:41:20.697548 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.697518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb\" (UID: \"eea9c2c1-1140-41c5-be33-7148d4560949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:20.798254 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.798221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb\" (UID: \"eea9c2c1-1140-41c5-be33-7148d4560949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:20.798875 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.798857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb\" (UID: \"eea9c2c1-1140-41c5-be33-7148d4560949\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:20.985565 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:20.985476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:21.125128 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:21.125101    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"]
Jul 28 21:41:21.126666 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:41:21.126637    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea9c2c1_1140_41c5_be33_7148d4560949.slice/crio-8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5 WatchSource:0}: Error finding container 8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5: Status 404 returned error can't find the container with id 8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5
Jul 28 21:41:21.951181 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:21.951130    2567 generic.go:356] "Generic (PLEG): container finished" podID="eea9c2c1-1140-41c5-be33-7148d4560949" containerID="dc2d64bc9fb234b2012422dd4b56f2c913a8728c1310117a6bd73a03e8240586" exitCode=0
Jul 28 21:41:21.951566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:21.951233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb" event={"ID":"eea9c2c1-1140-41c5-be33-7148d4560949","Type":"ContainerDied","Data":"dc2d64bc9fb234b2012422dd4b56f2c913a8728c1310117a6bd73a03e8240586"}
Jul 28 21:41:21.951566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:21.951317    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb" event={"ID":"eea9c2c1-1140-41c5-be33-7148d4560949","Type":"ContainerStarted","Data":"8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5"}
Jul 28 21:41:23.091079 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.091056    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:23.116103 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.116077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca\") pod \"eea9c2c1-1140-41c5-be33-7148d4560949\" (UID: \"eea9c2c1-1140-41c5-be33-7148d4560949\") "
Jul 28 21:41:23.116411 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.116389    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eea9c2c1-1140-41c5-be33-7148d4560949" (UID: "eea9c2c1-1140-41c5-be33-7148d4560949"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:41:23.217177 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.217064    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c2c1-1140-41c5-be33-7148d4560949-maas-api-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:23.963451 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.963417    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"
Jul 28 21:41:23.963643 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.963459    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb" event={"ID":"eea9c2c1-1140-41c5-be33-7148d4560949","Type":"ContainerDied","Data":"8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5"}
Jul 28 21:41:23.963643 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:23.963502    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8412935cf90e3f86c81103a0a82aef2ac33f3ed4e928d537801e97185a75b5c5"
Jul 28 21:41:25.154907 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.154873    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:41:25.155391 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.155117    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerName="maas-api" containerID="cri-o://2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161" gracePeriod=30
Jul 28 21:41:25.192916 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.192878    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.64:8443/health\": dial tcp 10.133.0.64:8443: connect: connection refused"
Jul 28 21:41:25.400964 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.400941    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:41:25.438187 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.438098    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca\") pod \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") "
Jul 28 21:41:25.438320 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.438196    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpg79\" (UniqueName: \"kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79\") pod \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") "
Jul 28 21:41:25.438320 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.438250    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls\") pod \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\" (UID: \"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33\") "
Jul 28 21:41:25.438526 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.438495    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" (UID: "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:41:25.440478 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.440451    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79" (OuterVolumeSpecName: "kube-api-access-rpg79") pod "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" (UID: "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33"). InnerVolumeSpecName "kube-api-access-rpg79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:41:25.440554 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.440495    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" (UID: "c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:41:25.539843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.539807    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpg79\" (UniqueName: \"kubernetes.io/projected/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-kube-api-access-rpg79\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:25.539843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.539836    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-maas-api-tls\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:25.539843 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.539845    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33-openshift-service-ca\") on node \"ip-10-0-140-162.ec2.internal\" DevicePath \"\""
Jul 28 21:41:25.610870 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.610835    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"]
Jul 28 21:41:25.613606 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.613576    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7076b1-nlwzb"]
Jul 28 21:41:25.975566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.975529    2567 generic.go:356] "Generic (PLEG): container finished" podID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerID="2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161" exitCode=0
Jul 28 21:41:25.975750 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.975577    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"
Jul 28 21:41:25.975750 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.975619    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" event={"ID":"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33","Type":"ContainerDied","Data":"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"}
Jul 28 21:41:25.975750 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.975671    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56" event={"ID":"c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33","Type":"ContainerDied","Data":"d94aab9d18cec2356268948fb4ba310e39d523c927cd00b71db625401a889781"}
Jul 28 21:41:25.975750 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.975692    2567 scope.go:117] "RemoveContainer" containerID="2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"
Jul 28 21:41:25.985204 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.985183    2567 scope.go:117] "RemoveContainer" containerID="2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"
Jul 28 21:41:25.985475 ip-10-0-140-162 kubenswrapper[2567]: E0728 21:41:25.985452    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161\": container with ID starting with 2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161 not found: ID does not exist" containerID="2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"
Jul 28 21:41:25.985575 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.985483    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161"} err="failed to get container status \"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161\": rpc error: code = NotFound desc = could not find container \"2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161\": container with ID starting with 2fa45489fcd7bf56cc6888d0e2965daab0e2e228a877a89cca64a9443d455161 not found: ID does not exist"
Jul 28 21:41:25.996945 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:25.996916    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:41:26.001081 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:26.001058    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7076b1-9f9d46d66-mbm56"]
Jul 28 21:41:26.076252 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:26.076223    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" path="/var/lib/kubelet/pods/c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33/volumes"
Jul 28 21:41:26.076549 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:26.076537    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eea9c2c1-1140-41c5-be33-7148d4560949" path="/var/lib/kubelet/pods/eea9c2c1-1140-41c5-be33-7148d4560949/volumes"
Jul 28 21:41:30.043394 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:30.043354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vfzvg"]
Jul 28 21:41:33.082763 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:33.082732    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd47df9db-lvlfm_fd714256-4d62-4f1e-ac14-37d45943d479/authorino/0.log"
Jul 28 21:41:39.420073 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:39.420043    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5966989555-pxtgx_a0c5f35e-9e99-4e02-931c-d4a311aa76a4/copy-manifests/0.log"
Jul 28 21:41:39.432380 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:39.432356    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5966989555-pxtgx_a0c5f35e-9e99-4e02-931c-d4a311aa76a4/manager/0.log"
Jul 28 21:41:40.043757 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:40.043730    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754555-lscm8_9ce35fc1-0e4c-4e59-9ef4-59ac43d6029a/cleanup/0.log"
Jul 28 21:41:40.154051 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:40.154023    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754570-wg4pz_5ff5a337-8a17-4132-881e-317babd7a653/cleanup/0.log"
Jul 28 21:41:41.086661 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.086631    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/util/0.log"
Jul 28 21:41:41.093497 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.093478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/pull/0.log"
Jul 28 21:41:41.098981 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.098962    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/extract/0.log"
Jul 28 21:41:41.219410 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.219382    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/util/0.log"
Jul 28 21:41:41.226402 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.226379    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/pull/0.log"
Jul 28 21:41:41.233319 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.233304    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/extract/0.log"
Jul 28 21:41:41.346618 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.346538    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/util/0.log"
Jul 28 21:41:41.353327 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.353305    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/pull/0.log"
Jul 28 21:41:41.360046 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.360026    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/extract/0.log"
Jul 28 21:41:41.476432 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.476398    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/pull/0.log"
Jul 28 21:41:41.482881 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.482859    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/extract/0.log"
Jul 28 21:41:41.488812 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.488795    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/util/0.log"
Jul 28 21:41:41.639751 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:41.639666    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd47df9db-lvlfm_fd714256-4d62-4f1e-ac14-37d45943d479/authorino/0.log"
Jul 28 21:41:42.008271 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:42.008193    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d46jc_a5d1c8c2-ff5e-4513-8b31-9c6da80d1399/kuadrant-console-plugin/0.log"
Jul 28 21:41:42.378383 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:42.378350    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vfzvg_38b0c4d7-bd28-486b-9960-3803a9d0e893/limitador/0.log"
Jul 28 21:41:43.102344 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:43.102321    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d6c74c55-45hwl_ebd954c0-7142-47a1-81d9-736a4a6890a2/kube-auth-proxy/0.log"
Jul 28 21:41:44.310154 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:44.310119    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7cj72_821e1404-344d-4f62-a156-130a80590e31/main/0.log"
Jul 28 21:41:50.252118 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252082    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-phbxh/must-gather-jw97p"]
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252450    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eea9c2c1-1140-41c5-be33-7148d4560949" containerName="revoke-keys"
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252463    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eea9c2c1-1140-41c5-be33-7148d4560949" containerName="revoke-keys"
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252482    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerName="maas-api"
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252487    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerName="maas-api"
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252555    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2596fa2-9fbc-4dfe-ba38-ae2800d1fb33" containerName="maas-api"
Jul 28 21:41:50.252580 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.252566    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eea9c2c1-1140-41c5-be33-7148d4560949" containerName="revoke-keys"
Jul 28 21:41:50.255899 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.255880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.258379 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.258360    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phbxh\"/\"openshift-service-ca.crt\""
Jul 28 21:41:50.258491 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.258474    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phbxh\"/\"kube-root-ca.crt\""
Jul 28 21:41:50.258564 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.258551    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-phbxh\"/\"default-dockercfg-xr4cn\""
Jul 28 21:41:50.264113 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.264092    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phbxh/must-gather-jw97p"]
Jul 28 21:41:50.348795 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.348759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8cwn\" (UniqueName: \"kubernetes.io/projected/a7a958ee-c85d-42aa-9168-79044bd89b86-kube-api-access-n8cwn\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.348970 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.348813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a7a958ee-c85d-42aa-9168-79044bd89b86-must-gather-output\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.450286 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.450254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8cwn\" (UniqueName: \"kubernetes.io/projected/a7a958ee-c85d-42aa-9168-79044bd89b86-kube-api-access-n8cwn\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.450498 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.450315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a7a958ee-c85d-42aa-9168-79044bd89b86-must-gather-output\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.450669 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.450648    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a7a958ee-c85d-42aa-9168-79044bd89b86-must-gather-output\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.461835 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.461814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8cwn\" (UniqueName: \"kubernetes.io/projected/a7a958ee-c85d-42aa-9168-79044bd89b86-kube-api-access-n8cwn\") pod \"must-gather-jw97p\" (UID: \"a7a958ee-c85d-42aa-9168-79044bd89b86\") " pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.565901 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.565871    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phbxh/must-gather-jw97p"
Jul 28 21:41:50.709593 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.709566    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phbxh/must-gather-jw97p"]
Jul 28 21:41:50.711039 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:41:50.711012    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7a958ee_c85d_42aa_9168_79044bd89b86.slice/crio-1e4c43a96a41f9a9c92b2aaf4e31968ed474ad4e48284cdb9e084574e52c02cd WatchSource:0}: Error finding container 1e4c43a96a41f9a9c92b2aaf4e31968ed474ad4e48284cdb9e084574e52c02cd: Status 404 returned error can't find the container with id 1e4c43a96a41f9a9c92b2aaf4e31968ed474ad4e48284cdb9e084574e52c02cd
Jul 28 21:41:50.713478 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:50.713459    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:41:51.114754 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:51.114718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phbxh/must-gather-jw97p" event={"ID":"a7a958ee-c85d-42aa-9168-79044bd89b86","Type":"ContainerStarted","Data":"1e4c43a96a41f9a9c92b2aaf4e31968ed474ad4e48284cdb9e084574e52c02cd"}
Jul 28 21:41:52.122389 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:52.122350    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phbxh/must-gather-jw97p" event={"ID":"a7a958ee-c85d-42aa-9168-79044bd89b86","Type":"ContainerStarted","Data":"e6e58fa0bbc32a87b279291f0838fdadeeaa02c5a03ca667af963d4efe3c0af3"}
Jul 28 21:41:52.122389 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:52.122396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phbxh/must-gather-jw97p" event={"ID":"a7a958ee-c85d-42aa-9168-79044bd89b86","Type":"ContainerStarted","Data":"b298580926510d0180647306df2d9c7061da230d7765e2b07bd0908f8fa95ead"}
Jul 28 21:41:52.139109 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:52.139062    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-phbxh/must-gather-jw97p" podStartSLOduration=1.2199616899999999 podStartE2EDuration="2.139047539s" totalImagesPullingTime="919.085849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:41:52.137810861 +0000 UTC m=+3406.536867526" watchObservedRunningTime="2026-07-28 21:41:52.139047539 +0000 UTC m=+3406.538104203"
Jul 28 21:41:53.273888 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:53.273858    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sqwjl_b23b1762-9707-464c-a2c1-22ae6e884d18/global-pull-secret-syncer/0.log"
Jul 28 21:41:53.432534 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:53.432495    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qvnqk_232a6fb4-b927-4db1-8957-56f14e6256c0/konnectivity-agent/0.log"
Jul 28 21:41:53.501978 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:53.501949    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-162.ec2.internal_66217b0fb27a8d11ff5105a5ffa34535/haproxy/0.log"
Jul 28 21:41:58.282792 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.282652    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/extract/0.log"
Jul 28 21:41:58.307863 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.307830    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/util/0.log"
Jul 28 21:41:58.335480 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.335430    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqr87_0e1d9fff-25af-4d25-834e-1d89f24fc889/pull/0.log"
Jul 28 21:41:58.369290 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.369259    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/extract/0.log"
Jul 28 21:41:58.396782 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.396757    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/util/0.log"
Jul 28 21:41:58.422417 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.422383    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x8tf5_83c9cff8-e09e-40e4-be58-70f725c0a8f7/pull/0.log"
Jul 28 21:41:58.465683 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.465634    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/extract/0.log"
Jul 28 21:41:58.493966 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.493920    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/util/0.log"
Jul 28 21:41:58.532971 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.532888    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732jrb4_38521bdd-0e98-416c-98f1-a8aaa3bfaf88/pull/0.log"
Jul 28 21:41:58.562862 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.562828    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/extract/0.log"
Jul 28 21:41:58.587152 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.587104    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/util/0.log"
Jul 28 21:41:58.618026 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:58.617990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ds87d_1fbfc9be-b99f-46fc-8e7b-c8e0363c6fd1/pull/0.log"
Jul 28 21:41:59.071593 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:59.071208    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd47df9db-lvlfm_fd714256-4d62-4f1e-ac14-37d45943d479/authorino/0.log"
Jul 28 21:41:59.155534 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:59.155490    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d46jc_a5d1c8c2-ff5e-4513-8b31-9c6da80d1399/kuadrant-console-plugin/0.log"
Jul 28 21:41:59.386770 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:41:59.386690    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vfzvg_38b0c4d7-bd28-486b-9960-3803a9d0e893/limitador/0.log"
Jul 28 21:42:01.036284 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.036239    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/alertmanager/0.log"
Jul 28 21:42:01.066126 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.066098    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/config-reloader/0.log"
Jul 28 21:42:01.090461 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.090380    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/kube-rbac-proxy-web/0.log"
Jul 28 21:42:01.112566 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.112508    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/kube-rbac-proxy/0.log"
Jul 28 21:42:01.135931 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.135898    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/kube-rbac-proxy-metric/0.log"
Jul 28 21:42:01.160370 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.160334    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/prom-label-proxy/0.log"
Jul 28 21:42:01.183605 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.183578    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc04b02-32c0-4e69-b872-bb97fc3b58cf/init-config-reloader/0.log"
Jul 28 21:42:01.273945 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.273880    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rvxwz_45988f53-82ef-4c7a-8d00-edf881f30ce2/kube-state-metrics/0.log"
Jul 28 21:42:01.295249 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.295164    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rvxwz_45988f53-82ef-4c7a-8d00-edf881f30ce2/kube-rbac-proxy-main/0.log"
Jul 28 21:42:01.327157 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.326440    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rvxwz_45988f53-82ef-4c7a-8d00-edf881f30ce2/kube-rbac-proxy-self/0.log"
Jul 28 21:42:01.387433 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.387355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-rb94r_95b7ef76-3e7e-4db2-b19f-7775fe07b4d6/monitoring-plugin/0.log"
Jul 28 21:42:01.500825 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.500791    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj8lf_c7d3ad98-23b1-4bda-a877-10b3845af57d/node-exporter/0.log"
Jul 28 21:42:01.523471 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.523436    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj8lf_c7d3ad98-23b1-4bda-a877-10b3845af57d/kube-rbac-proxy/0.log"
Jul 28 21:42:01.547276 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.547245    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj8lf_c7d3ad98-23b1-4bda-a877-10b3845af57d/init-textfile/0.log"
Jul 28 21:42:01.959030 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.958999    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-vbrf4_495e1672-c7fe-4d51-921c-d44eee1f1b50/prometheus-operator/0.log"
Jul 28 21:42:01.977307 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:01.977258    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-vbrf4_495e1672-c7fe-4d51-921c-d44eee1f1b50/kube-rbac-proxy/0.log"
Jul 28 21:42:02.004512 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.004459    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-mz5xb_73e05204-14d9-4285-8a11-1d4e86224fd9/prometheus-operator-admission-webhook/0.log"
Jul 28 21:42:02.037255 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.037229    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c5c69fbdd-dv8k8_7603053a-c958-4c92-b291-aa103a1c62dd/telemeter-client/0.log"
Jul 28 21:42:02.078899 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.078868    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c5c69fbdd-dv8k8_7603053a-c958-4c92-b291-aa103a1c62dd/reload/0.log"
Jul 28 21:42:02.102685 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.102658    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c5c69fbdd-dv8k8_7603053a-c958-4c92-b291-aa103a1c62dd/kube-rbac-proxy/0.log"
Jul 28 21:42:02.136867 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.136841    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/thanos-query/0.log"
Jul 28 21:42:02.159823 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.159748    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/kube-rbac-proxy-web/0.log"
Jul 28 21:42:02.182827 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.182792    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/kube-rbac-proxy/0.log"
Jul 28 21:42:02.203978 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.203950    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/prom-label-proxy/0.log"
Jul 28 21:42:02.257076 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.257042    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/kube-rbac-proxy-rules/0.log"
Jul 28 21:42:02.279525 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:02.279487    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bc9f8bb7-scjpc_f56d3951-e4ef-4091-af09-961b4837a911/kube-rbac-proxy-metrics/0.log"
Jul 28 21:42:03.424667 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.424634    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"]
Jul 28 21:42:03.430356 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.430329    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.435432 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.435405    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"]
Jul 28 21:42:03.603384 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.603344    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-podres\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.603755 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.603730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-proc\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.603925 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.603907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmhs\" (UniqueName: \"kubernetes.io/projected/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-kube-api-access-vvmhs\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.604058 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.604045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-lib-modules\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.604285 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.604267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-sys\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.705794 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.705707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-proc\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706052 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706024    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmhs\" (UniqueName: \"kubernetes.io/projected/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-kube-api-access-vvmhs\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706232 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-lib-modules\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706421 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706408    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-sys\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706549 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-podres\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706711 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-podres\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706790 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706592    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-sys\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706790 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.705946    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-proc\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.706790 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.706549    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-lib-modules\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.718690 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.718660    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmhs\" (UniqueName: \"kubernetes.io/projected/61b5589a-b4c1-41d9-9e9e-330dc79daf8f-kube-api-access-vvmhs\") pod \"perf-node-gather-daemonset-kkz85\" (UID: \"61b5589a-b4c1-41d9-9e9e-330dc79daf8f\") " pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.744498 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.744460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:03.951653 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:03.951622    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"]
Jul 28 21:42:03.955202 ip-10-0-140-162 kubenswrapper[2567]: W0728 21:42:03.955172    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61b5589a_b4c1_41d9_9e9e_330dc79daf8f.slice/crio-43e9a98ec841060d293fb34b03d6a13c01bc2ad3119fde2b686b41198fc202c2 WatchSource:0}: Error finding container 43e9a98ec841060d293fb34b03d6a13c01bc2ad3119fde2b686b41198fc202c2: Status 404 returned error can't find the container with id 43e9a98ec841060d293fb34b03d6a13c01bc2ad3119fde2b686b41198fc202c2
Jul 28 21:42:04.231530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:04.231436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85" event={"ID":"61b5589a-b4c1-41d9-9e9e-330dc79daf8f","Type":"ContainerStarted","Data":"8de230a8dce8cba4fbdd0cf9975caa475f14bef170535c4428f4492a475c060e"}
Jul 28 21:42:04.231530 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:04.231517    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85" event={"ID":"61b5589a-b4c1-41d9-9e9e-330dc79daf8f","Type":"ContainerStarted","Data":"43e9a98ec841060d293fb34b03d6a13c01bc2ad3119fde2b686b41198fc202c2"}
Jul 28 21:42:04.231764 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:04.231687    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:04.251952 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:04.251888    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85" podStartSLOduration=1.2518657690000001 podStartE2EDuration="1.251865769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 21:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:42:04.248784332 +0000 UTC m=+3418.647840998" watchObservedRunningTime="2026-07-28 21:42:04.251865769 +0000 UTC m=+3418.650922435"
Jul 28 21:42:04.634673 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:04.634644    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d4b79bd4-wlzbq_5c1e1b9b-fa61-4f77-a19e-0e912e5101cd/console/0.log"
Jul 28 21:42:06.336628 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:06.336597    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4cfcq_98db2d83-d062-4737-8e53-c31ba8cb72ad/dns/0.log"
Jul 28 21:42:06.358968 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:06.358942    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4cfcq_98db2d83-d062-4737-8e53-c31ba8cb72ad/kube-rbac-proxy/0.log"
Jul 28 21:42:06.499355 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:06.499324    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fkqs2_c1c60473-776e-49c5-970f-91b2e51f8f4b/dns-node-resolver/0.log"
Jul 28 21:42:07.128268 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:07.128242    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9fkq9_a9485831-57b2-4234-9516-63fdc259e799/node-ca/0.log"
Jul 28 21:42:08.293263 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:08.293228    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d6c74c55-45hwl_ebd954c0-7142-47a1-81d9-736a4a6890a2/kube-auth-proxy/0.log"
Jul 28 21:42:09.250943 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:09.250914    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cldn8_c35c4f86-4695-46f1-911e-f4bdb215b4f1/serve-healthcheck-canary/0.log"
Jul 28 21:42:09.886724 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:09.886695    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ddf7_514b3bd1-2f12-4d4a-b437-63d4e629fcfd/kube-rbac-proxy/0.log"
Jul 28 21:42:09.910072 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:09.910044    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ddf7_514b3bd1-2f12-4d4a-b437-63d4e629fcfd/exporter/0.log"
Jul 28 21:42:09.942611 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:09.942584    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ddf7_514b3bd1-2f12-4d4a-b437-63d4e629fcfd/extractor/0.log"
Jul 28 21:42:10.249082 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:10.249008    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-phbxh/perf-node-gather-daemonset-kkz85"
Jul 28 21:42:13.212097 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:13.212062    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5966989555-pxtgx_a0c5f35e-9e99-4e02-931c-d4a311aa76a4/manager/0.log"
Jul 28 21:42:13.232018 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:13.231993    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5966989555-pxtgx_a0c5f35e-9e99-4e02-931c-d4a311aa76a4/copy-manifests/0.log"
Jul 28 21:42:14.200989 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:14.200956    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d6c74c55-45hwl_ebd954c0-7142-47a1-81d9-736a4a6890a2/kube-auth-proxy/0.log"
Jul 28 21:42:14.514620 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:14.514593    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57d8fcb7f5-czwbc_6cf0c7ca-a908-40df-ac19-e44cf765175b/manager/0.log"
Jul 28 21:42:15.853346 ip-10-0-140-162 kubenswrapper[2567]: I0728 21:42:15.853309    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d6c74c55-45hwl_ebd954c0-7142-47a1-81d9-736a4a6890a2/kube-auth-proxy/0.log"