Jul 06 14:22:12.847572 ip-10-0-140-204 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 14:22:12.847741 ip-10-0-140-204 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 14:22:12.847758 ip-10-0-140-204 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 14:22:12.848188 ip-10-0-140-204 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 14:22:22.873434 ip-10-0-140-204 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 14:22:22.873450 ip-10-0-140-204 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b4b17f4b859d46cd824c1b4ff1dc3238 --
Jul 06 14:24:09.670470 ip-10-0-140-204 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.066085    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070091    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070105    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070108    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070111    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070114    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070117    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070119    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.079148 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070122    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070125    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070128    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070131    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070133    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070136    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070139    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070142    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070144    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070147    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070151    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070154    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070156    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070159    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070161    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070165    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070167    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070170    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070174    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070176    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070179    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.079986 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070182    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070187    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070191    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070195    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070198    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070202    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070215    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070219    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070222    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070225    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070228    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070231    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070234    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070237    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070240    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070243    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070246    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070249    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070252    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.080616 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070255    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070257    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070261    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070264    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070266    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070269    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070271    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070274    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070276    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070279    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070283    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070285    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070288    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070291    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070294    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070296    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070299    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070303    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070306    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.081238 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070308    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070311    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070315    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070318    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070321    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070324    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070328    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070332    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070335    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070338    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070340    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070343    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070346    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070349    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070351    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070354    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070357    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070360    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070362    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.081847 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070364    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070367    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070369    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070372    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070375    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070378    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070380    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070383    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070385    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070387    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070390    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070393    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070395    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.070397    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073525    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073534    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073537    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073540    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073543    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073546    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.082461 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073549    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073552    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073554    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073557    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073560    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073565    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073569    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073573    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073575    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073578    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073586    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073589    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073592    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073595    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073597    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073600    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073603    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073605    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073608    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073611    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.083304 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073613    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073616    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073619    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073622    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073624    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073627    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073631    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073634    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073637    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073639    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073642    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073644    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073648    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073651    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073653    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073656    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073658    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073661    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073664    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073667    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.083936 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073669    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073674    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073678    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073689    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073692    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073695    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073698    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073701    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073704    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073706    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073709    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073712    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073714    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073717    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073719    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073722    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073724    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073727    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073730    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.084575 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073732    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073735    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073740    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073742    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073745    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073747    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073750    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073754    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073756    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073760    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073762    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073765    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073768    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073771    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073773    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073776    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073779    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073787    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073790    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073792    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.085246 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073794    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073797    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073799    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073802    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073804    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073807    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073810    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073813    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073815    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073818    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073820    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073823    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073825    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.073828    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074327    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074333    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074336    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074339    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074342    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074344    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.085885 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074347    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074350    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074352    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074356    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074360    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074363    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074365    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074369    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074372    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074381    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074384    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074387    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074389    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074392    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074395    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074398    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074401    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074404    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074406    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.086558 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074409    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074411    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074414    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074416    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074419    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074421    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074424    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074429    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074431    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074433    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074436    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074438    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074441    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074443    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074446    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074448    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074451    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074453    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074456    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074458    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.087222 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074462    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074464    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074467    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074475    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074478    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074480    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074483    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074485    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074488    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074490    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074493    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074495    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074497    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074500    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074502    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074505    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074507    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074510    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074512    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.090039 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074516    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074521    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074525    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074527    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074530    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074533    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074535    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074538    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074541    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074543    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074546    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074548    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074551    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074553    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074556    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074558    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074561    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074569    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074571    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074574    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.090673 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074576    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074578    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074581    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074583    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074586    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074588    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074590    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074593    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074595    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074598    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074600    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074603    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074611    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074613    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.074616    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074682    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074689    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074698    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074703    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074708    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074711    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 14:24:10.193033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074715    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 14:24:10.144298 ip-10-0-140-204 systemd[1]: Started Kubernetes Kubelet.
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074720    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074723    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074726    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074729    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074733    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074736    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074739    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074742    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074750    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074754    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074756    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074763    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074767    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074770    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074773    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074776    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074780    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074783    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074786    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074790    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074793    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074795    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074800    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 14:24:10.194106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074803    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074806    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074811    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074814    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074817    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074820    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074823    2570 flags.go:64] FLAG: --enable-server="true"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074826    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074832    2570 flags.go:64] FLAG: --event-burst="100"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074836    2570 flags.go:64] FLAG: --event-qps="50"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074838    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074841    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074844    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074848    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074851    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074854    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074857    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074859    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074869    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074872    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074875    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074878    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074881    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074884    2570 flags.go:64] FLAG: --feature-gates=""
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074887    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 14:24:10.194904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074891    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074894    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074897    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074900    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074902    2570 flags.go:64] FLAG: --help="false"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074905    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074910    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074913    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074916    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074921    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074924    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074927    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074930    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074933    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074936    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074939    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074942    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074945    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074948    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074950    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074953    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074956    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074959    2570 flags.go:64] FLAG: --lock-file=""
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074962    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 14:24:10.195666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074965    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074967    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074978    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074981    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.074984    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075002    2570 flags.go:64] FLAG: --logging-format="text"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075006    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075009    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075012    2570 flags.go:64] FLAG: --manifest-url=""
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075015    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075019    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075023    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075027    2570 flags.go:64] FLAG: --max-pods="110"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075030    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075034    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075043    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075046    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075050    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075053    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075056    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075063    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075066    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075069    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 14:24:10.196373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075073    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075075    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075081    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075084    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075087    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075090    2570 flags.go:64] FLAG: --port="10250"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075093    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075096    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eeb1f9d26d27b087"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075099    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075102    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075105    2570 flags.go:64] FLAG: --register-node="true"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075108    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075118    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075121    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075123    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075126    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075130    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075133    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075136    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075139    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075142    2570 flags.go:64] FLAG: --runonce="false"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075145    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075147    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075151    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075154    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 14:24:10.197044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075157    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075160    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075165    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075168    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075171    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075174    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075177    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075180    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075183    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075185    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075188    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075194    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075197    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075200    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075207    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075209    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075212    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075215    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075218    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075221    2570 flags.go:64] FLAG: --v="2"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075230    2570 flags.go:64] FLAG: --version="false"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075234    2570 flags.go:64] FLAG: --vmodule=""
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075237    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075241    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075440    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.197860 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075444    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075447    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075450    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075453    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075455    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075461    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075463    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075466    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075469    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075473    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075475    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075478    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075481    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075483    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075486    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075490    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075494    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075497    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075499    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.198633 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075502    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075504    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075506    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075509    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075512    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075514    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075517    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075521    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075525    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075534    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075537    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075540    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075543    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075545    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075548    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075550    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075553    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075555    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075560    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.199458 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075562    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075565    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075567    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075571    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075574    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075576    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075579    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075581    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075584    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075586    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075589    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075592    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075605    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075608    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075611    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075614    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075616    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075619    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075622    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075624    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075626    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.200434 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075629    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075632    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075640    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075642    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075645    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075647    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075650    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075652    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075655    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075657    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075661    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075664    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075667    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075670    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075673    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075676    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075678    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075681    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075683    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075686    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.201574 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075689    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075692    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075694    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075697    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075699    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075702    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075704    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075707    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075709    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075712    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075714    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075717    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075719    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075722    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075724    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075730    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075733    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075735    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.202680 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.075738    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.075746    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.087674    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.087695    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.088574    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.088584    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088662    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088669    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088674    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088679    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088683    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088688    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088691    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088695    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088699    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.203642 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088703    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088707    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088712    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088716    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088720    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088724    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088729    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088733    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088737    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088742    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088746    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088749    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088753    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088757    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088761    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088765    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088770    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088774    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088778    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.204403 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088782    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088786    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088790    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088794    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088798    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088802    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088806    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088812    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088818    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088823    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088827    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088831    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088835    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088839    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088842    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088847    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088850    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088854    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088858    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088862    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.205154 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088867    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088871    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088874    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088878    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088882    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088887    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088891    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088895    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088899    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088903    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088908    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088912    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088916    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088919    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088923    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088927    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088931    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088934    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088937    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088941    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.205730 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088945    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088949    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088953    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088957    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088961    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088965    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088969    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088972    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088976    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088980    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.088985    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089001    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089005    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089010    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089014    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089018    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089023    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089029    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089033    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.206263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089036    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089041    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089045    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089049    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089053    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089057    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089061    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089065    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089069    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089072    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089078    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089083    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.089090    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089252    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:24:10.206807 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089259    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089263    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089268    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089271    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089276    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089280    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089284    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089288    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089292    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089296    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089300    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089305    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089309    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089313    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089317    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089321    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089325    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089328    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089333    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089337    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:24:10.207201 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089340    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089344    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089348    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089352    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089356    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089360    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089364    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089368    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089372    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089376    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089380    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089383    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089387    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089391    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089395    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089400    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089403    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089407    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089411    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089416    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:24:10.207754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089421    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089424    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089428    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089432    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089437    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089441    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089445    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089449    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089453    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089456    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089460    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089464    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089468    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089472    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089475    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089479    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089483    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089487    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089491    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089495    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:24:10.208698 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089498    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089504    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089509    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089513    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089517    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089521    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089525    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089528    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089532    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089536    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089539    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089543    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089547    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089551    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089554    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089558    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089564    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089570    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089574    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:24:10.209360 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089579    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089583    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089587    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089591    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089595    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089599    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089602    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089607    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089613    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089617    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089621    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089624    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089629    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089632    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089637    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089641    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089644    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089653    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:24:10.352033 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:10.089657    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.089665    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.089875    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.092481    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.093786    2570 server.go:1019] "Starting client certificate rotation"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.093894    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.093930    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.096638    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.102087    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.109579    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.114703    2570 log.go:25] "Validated CRI v1 image API"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.116078    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 14:24:10.366124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.118769    2570 fs.go:135] Filesystem UUIDs: map[39817b7c-5604-47bf-ac17-df9fdccd0cce:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f3d2980a-1f7e-48b5-a38d-7fa74b2418d5:/dev/nvme0n1p4]
Jul 06 14:24:10.366504 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.118787    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.125073    2570 manager.go:216] Machine: {Timestamp:2026-07-06 14:24:10.123714632 +0000 UTC m=+0.332523025 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100596 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261b3888a170ff7936397c0f48d958 SystemUUID:ec261b38-88a1-70ff-7936-397c0f48d958 BootID:b4b17f4b-859d-46cd-824c-1b4ff1dc3238 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:3f:b4:f0:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:3f:b4:f0:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:ef:75:f5:50:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.125173    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.125260    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.125834    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.127736    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.127765    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 14:24:10.366603 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.128935    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.128946    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.128954    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.129568    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.130705    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.133874    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134035    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134067    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134079    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134093    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134104    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134112    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134536    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.134553    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.136714    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.138107    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.138133    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139437    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139452    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139459    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139464    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139472    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139477    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139485    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139490    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139500    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139505    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139516    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.139522    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.140143    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.140149    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.143386    2570 server.go:1268] "Started kubelet"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.146388    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.146505    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.146707    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.147875    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.149756    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.150616    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.150672    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 14:24:10.366844 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.150747    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.150593    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-204.ec2.internal.18bfb96bcec0136f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-204.ec2.internal,UID:ip-10-0-140-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-204.ec2.internal,},FirstTimestamp:2026-07-06 14:24:10.143339375 +0000 UTC m=+0.352147794,LastTimestamp:2026-07-06 14:24:10.143339375 +0000 UTC m=+0.352147794,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-204.ec2.internal,}"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.151650    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153037    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153061    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.153180    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153628    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153641    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153748    2570 factory.go:55] Registering systemd factory
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153757    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153763    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153770    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.153779    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154012    2570 factory.go:153] Registering CRI-O factory
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154021    2570 factory.go:223] Registration of the crio container factory successfully
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154061    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154082    2570 factory.go:103] Registering Raw factory
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154092    2570 manager.go:1196] Started watching for new ooms in manager
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.154483    2570 manager.go:349] Starting recovery of all containers
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.155303    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.155545    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.161542    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.163896    2570 manager.go:354] Recovery completed
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.168252    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170094    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170124    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170135    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170650    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170661    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170677    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.170663    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171373    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171403    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.367919 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171416    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171854    2570 policy_none.go:49] "None policy: Start"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171870    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.171893    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.172963    2570 policy_none.go:47] "Start"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.177541    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pfctl"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.181057    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pfctl"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.182721    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-204.ec2.internal.18bfb96bd05881ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-204.ec2.internal,UID:ip-10-0-140-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-204.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-204.ec2.internal,},FirstTimestamp:2026-07-06 14:24:10.170106314 +0000 UTC m=+0.378914689,LastTimestamp:2026-07-06 14:24:10.170106314 +0000 UTC m=+0.378914689,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-204.ec2.internal,}"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.199707    2570 manager.go:337] "Starting Device Plugin manager"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.199744    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.199756    2570 server.go:87] "Starting device plugin registration server"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.200047    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.200058    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.200144    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.200207    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.200213    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.200730    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.200774    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.286099    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.286136    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.286159    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.286194    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.287867    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.300527    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.301479    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.301505    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.301519    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.301549    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.307253    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.307276    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-204.ec2.internal\": node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.368786 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.334346    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.387256    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"]
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.387341    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.388303    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.388331    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.388343    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.389729    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.389844    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.389871    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390414    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390452    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390474    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390456    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390489    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.390504    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.391699    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.391720    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.392395    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.392419    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.392432    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.422822    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-204.ec2.internal\" not found" node="ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.427310    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-204.ec2.internal\" not found" node="ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.434705    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.456533    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.456559    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.506592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.456580    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ee29cf4a0b9a74c02a59c261bf9c8fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-204.ec2.internal\" (UID: \"7ee29cf4a0b9a74c02a59c261bf9c8fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.535130    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557370    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557414    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ee29cf4a0b9a74c02a59c261bf9c8fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-204.ec2.internal\" (UID: \"7ee29cf4a0b9a74c02a59c261bf9c8fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557467    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557475    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b8b6363e2d727a9190ed846d2b0458-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal\" (UID: \"90b8b6363e2d727a9190ed846d2b0458\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.557466    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ee29cf4a0b9a74c02a59c261bf9c8fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-204.ec2.internal\" (UID: \"7ee29cf4a0b9a74c02a59c261bf9c8fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.642292 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.635569    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.778211 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.725477    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.778211 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:10.730054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:10.778211 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.736165    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:10.921919 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.836569    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.072760 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:10.937051    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.072760 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.037482    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.092914    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.093095    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.138543    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.139710    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.182850    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:19:10 +0000 UTC" deadline="2028-03-14 12:35:37.160504162 +0000 UTC"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.182900    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14806h11m25.97760734s"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.199983    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.200979    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.201024    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:24:11.219481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.201032    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:24:11.314879 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.238936    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:11.338930    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b8b6363e2d727a9190ed846d2b0458.slice/crio-8dc57a63602f48171ed7a0927c8489535458f0ec89cf69e13300235679c885d1 WatchSource:0}: Error finding container 8dc57a63602f48171ed7a0927c8489535458f0ec89cf69e13300235679c885d1: Status 404 returned error can't find the container with id 8dc57a63602f48171ed7a0927c8489535458f0ec89cf69e13300235679c885d1
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.338986    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:11.340935    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee29cf4a0b9a74c02a59c261bf9c8fd.slice/crio-e732be6e56d6756f10ca4e6d148d75d4e750cff88091c6b66348c7fe04b160a6 WatchSource:0}: Error finding container e732be6e56d6756f10ca4e6d148d75d4e750cff88091c6b66348c7fe04b160a6: Status 404 returned error can't find the container with id e732be6e56d6756f10ca4e6d148d75d4e750cff88091c6b66348c7fe04b160a6
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.347703    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.369737    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:24:11.446784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.440045    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:11.540491    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-204.ec2.internal\" not found"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.548955    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.553417    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.566360    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.567236    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal"
Jul 06 14:24:11.588635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:11.576220    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 14:24:12.135135 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.134818    2570 apiserver.go:52] "Watching apiserver"
Jul 06 14:24:12.146845 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.146806    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 14:24:12.148368 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.148274    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-hf2kl","kube-system/global-pull-secret-syncer-wlxc7","openshift-image-registry/node-ca-v6vdx","openshift-multus/multus-ggh7s","openshift-multus/network-metrics-daemon-smh7l","openshift-network-operator/iptables-alerter-2wrmd","openshift-ovn-kubernetes/ovnkube-node-jxdnx","kube-system/konnectivity-agent-d8xgh","kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq","openshift-cluster-node-tuning-operator/tuned-xd45t","openshift-dns/node-resolver-b9cph","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal","openshift-multus/multus-additional-cni-plugins-522sb"]
Jul 06 14:24:12.150588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.150361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.152340 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.152321    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.153797 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.153769    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5thkm\""
Jul 06 14:24:12.154402 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.154044    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 14:24:12.154725 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.154700    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.155017 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.154980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.156871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.155686    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 14:24:12.156871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.155894    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.156871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.155693    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 14:24:12.156871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.156549    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.157956 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.157836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.158066 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.157911    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:12.160038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.159296    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.161099 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.161080    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vzzgw\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.161331    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.161380    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.161539    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162275    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162546    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162552    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.162759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162620    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 14:24:12.163198 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162779    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.163198 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.162911    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.163294 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.163266    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.164195 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164172    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 14:24:12.164339 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164319    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c9xdf\""
Jul 06 14:24:12.164697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164378    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.164781 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164478    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pqgxw\""
Jul 06 14:24:12.164781 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.164898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164538    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f9m7z\""
Jul 06 14:24:12.164898 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.164872    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:12.164898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164593    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 14:24:12.165079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164641    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 14:24:12.165079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.164590    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 14:24:12.165282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165263    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.165607 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165587    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vgmrw\""
Jul 06 14:24:12.165607 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165599    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-host\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.165730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165629    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b85th\" (UniqueName: \"kubernetes.io/projected/23730560-a85b-48ec-b576-1b8164de769e-kube-api-access-b85th\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.165730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-serviceca\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.165730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165678    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-kubelet\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165703    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-systemd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-config\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165796    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-script-lib\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165821    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-var-lib-kubelet\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165844    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-var-lib-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165849    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-bin\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-netd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.165923 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2mlh\" (UniqueName: \"kubernetes.io/projected/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-kube-api-access-s2mlh\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysconfig\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.165964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-host\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-env-overrides\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166036    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovn-node-metrics-cert\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166043    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166061    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzj6t\" (UniqueName: \"kubernetes.io/projected/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-kube-api-access-wzj6t\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166094    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-netns\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166118    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166150    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-ovn\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166173    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-kubernetes\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-conf\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166219    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-systemd\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-lib-modules\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.166365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166273    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-log-socket\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166311    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-systemd-units\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166322    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166334    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-node-log\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166368    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166393    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-modprobe-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166414    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166435    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-sys\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166465    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-etc-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166501    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-etc-tuned\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166529    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-tmp\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166553    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-slash\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.167069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.166576    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-run\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.169803 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.169102    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:12.169803 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.169186    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:12.171176 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.170739    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hdqfq\""
Jul 06 14:24:12.171176 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.171063    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 14:24:12.171319 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.171236    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 14:24:12.172565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.171593    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wh6cl\""
Jul 06 14:24:12.172565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.172057    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.172565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.172134    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.172565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.172373    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 14:24:12.173668 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.173463    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.178108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.178088    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 14:24:12.178214 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.178199    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-skqzn\""
Jul 06 14:24:12.178470 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.178453    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 14:24:12.178551 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.178454    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 14:24:12.255118 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.255093    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 14:24:12.267364 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhhmb\" (UniqueName: \"kubernetes.io/projected/4225a046-a4d4-498f-a7e6-587bd1fd6017-kube-api-access-zhhmb\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-host\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267404    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b85th\" (UniqueName: \"kubernetes.io/projected/23730560-a85b-48ec-b576-1b8164de769e-kube-api-access-b85th\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-serviceca\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267444    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-kubelet\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267465    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-systemd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-script-lib\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfbpp\" (UniqueName: \"kubernetes.io/projected/fc515b02-bb79-4f8d-9122-ca6043180ae8-kube-api-access-mfbpp\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267565    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-netd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267600    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2mlh\" (UniqueName: \"kubernetes.io/projected/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-kube-api-access-s2mlh\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267628    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysconfig\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267652    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-host\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-sys-fs\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-hostroot\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267762    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-conf-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267787    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgzv4\" (UniqueName: \"kubernetes.io/projected/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-kube-api-access-hgzv4\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267810    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-host\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovn-node-metrics-cert\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.267857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267850    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-socket-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267874    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-netd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc515b02-bb79-4f8d-9122-ca6043180ae8-hosts-file\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.267917    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-system-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268035    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-systemd\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268037    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysconfig\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268059    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-host\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268085    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wf8l\" (UniqueName: \"kubernetes.io/projected/542a6603-94a1-4a3b-afc1-589dbbdb7e27-kube-api-access-9wf8l\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268101    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268118    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-netns\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268143    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-kubelet\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268180    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-netns\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268201    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268206    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-ovn\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.268460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268246    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-run-ovn\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-kubernetes\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268369    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-conf\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-kubelet-config\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268381    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268432    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-serviceca\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268590    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-conf\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268597    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-kubernetes\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268700    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-dbus\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-device-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc515b02-bb79-4f8d-9122-ca6043180ae8-tmp-dir\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268784    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-script-lib\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268807    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-netns\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268833    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-multus\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268860    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.269184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268882    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542a6603-94a1-4a3b-afc1-589dbbdb7e27-host-slash\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-cnibin\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.268969    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-tuning-conf-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269019    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-systemd-units\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269044    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-node-log\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269068    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-modprobe-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269105    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-systemd-units\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269113    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-sys\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269143    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-node-log\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269151    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269181    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-bin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269211    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-etc-tuned\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269219    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-modprobe-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269234    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-tmp\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.269825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269266    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-sys\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269267    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-agent-certs\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-cnibin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269325    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-522xn\" (UniqueName: \"kubernetes.io/projected/f0f71116-392c-4b1f-888c-1995314b1cd2-kube-api-access-522xn\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269352    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-slash\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269375    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-run\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269405    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-config\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269423    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-slash\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269427    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-var-lib-kubelet\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269466    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269484    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-run\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269491    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-var-lib-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269516    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-bin\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269530    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-var-lib-kubelet\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.270545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269540    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-registration-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269698    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-daemon-config\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269702    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-var-lib-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269733    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-host-cni-bin\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-etc-kubernetes\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269820    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-env-overrides\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.269879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzj6t\" (UniqueName: \"kubernetes.io/projected/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-kube-api-access-wzj6t\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270053    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovnkube-config\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270233    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-env-overrides\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270418    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-multus-certs\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-systemd\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270516    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-lib-modules\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-k8s-cni-cncf-io\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270567    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-log-socket\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270586    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-systemd\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-konnectivity-ca\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270615    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270640    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270664    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tstbr\" (UniqueName: \"kubernetes.io/projected/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kube-api-access-tstbr\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270686    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-os-release\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270698    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-lib-modules\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270712    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-kubelet\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270734    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-system-cni-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270756    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270779    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270803    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-os-release\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.271858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270826    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-binary-copy\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270849    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270872    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-etc-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270918    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-cni-binary-copy\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270941    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-socket-dir-parent\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.270963    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/542a6603-94a1-4a3b-afc1-589dbbdb7e27-iptables-alerter-script\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.271083    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-log-socket\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.271173    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-etc-openvswitch\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.272486 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.271252    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/23730560-a85b-48ec-b576-1b8164de769e-etc-sysctl-d\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.272887 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.272803    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-tmp\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.272887 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.272862    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-ovn-node-metrics-cert\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.275109 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.275064    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/23730560-a85b-48ec-b576-1b8164de769e-etc-tuned\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.277783 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.277710    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b85th\" (UniqueName: \"kubernetes.io/projected/23730560-a85b-48ec-b576-1b8164de769e-kube-api-access-b85th\") pod \"tuned-xd45t\" (UID: \"23730560-a85b-48ec-b576-1b8164de769e\") " pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.278924 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.278763    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzj6t\" (UniqueName: \"kubernetes.io/projected/c6ed3cfe-e58e-4b15-ad73-2c821535d27d-kube-api-access-wzj6t\") pod \"node-ca-v6vdx\" (UID: \"c6ed3cfe-e58e-4b15-ad73-2c821535d27d\") " pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.279202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.279185    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2mlh\" (UniqueName: \"kubernetes.io/projected/7ac49d6c-415f-4ab6-9cfe-2809aac3423e-kube-api-access-s2mlh\") pod \"ovnkube-node-jxdnx\" (UID: \"7ac49d6c-415f-4ab6-9cfe-2809aac3423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.282903 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.282874    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jk64x"
Jul 06 14:24:12.290783 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.290741    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal" event={"ID":"7ee29cf4a0b9a74c02a59c261bf9c8fd","Type":"ContainerStarted","Data":"e732be6e56d6756f10ca4e6d148d75d4e750cff88091c6b66348c7fe04b160a6"}
Jul 06 14:24:12.291955 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.291925    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal" event={"ID":"90b8b6363e2d727a9190ed846d2b0458","Type":"ContainerStarted","Data":"8dc57a63602f48171ed7a0927c8489535458f0ec89cf69e13300235679c885d1"}
Jul 06 14:24:12.292287 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.292272    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jk64x"
Jul 06 14:24:12.371282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-bin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-agent-certs\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-cnibin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-522xn\" (UniqueName: \"kubernetes.io/projected/f0f71116-392c-4b1f-888c-1995314b1cd2-kube-api-access-522xn\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371366    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371391    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-registration-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-daemon-config\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371418    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-cnibin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-etc-kubernetes\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371468    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-etc-kubernetes\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371482    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-multus-certs\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371540    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-k8s-cni-cncf-io\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371565    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-konnectivity-ca\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371589    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371611    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371638    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tstbr\" (UniqueName: \"kubernetes.io/projected/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kube-api-access-tstbr\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371652    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-registration-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371662    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-os-release\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371663    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-k8s-cni-cncf-io\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371698    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-kubelet\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371714    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-multus-certs\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371721    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-os-release\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.371981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-system-cni-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371757    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.371849    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.371913    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:12.871890082 +0000 UTC m=+3.080698465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371911    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371363    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-bin\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371951    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.371970    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-kubelet\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372037    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-system-cni-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-os-release\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372213    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-binary-copy\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372281    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-os-release\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372316    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-konnectivity-ca\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372316    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.372759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372319    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372380    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-daemon-config\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-cni-binary-copy\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-socket-dir-parent\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/542a6603-94a1-4a3b-afc1-589dbbdb7e27-iptables-alerter-script\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372481    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhhmb\" (UniqueName: \"kubernetes.io/projected/4225a046-a4d4-498f-a7e6-587bd1fd6017-kube-api-access-zhhmb\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfbpp\" (UniqueName: \"kubernetes.io/projected/fc515b02-bb79-4f8d-9122-ca6043180ae8-kube-api-access-mfbpp\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372537    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-sys-fs\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372576    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-hostroot\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372590    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-sys-fs\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372601    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-conf-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372631    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgzv4\" (UniqueName: \"kubernetes.io/projected/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-kube-api-access-hgzv4\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-socket-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.373456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc515b02-bb79-4f8d-9122-ca6043180ae8-hosts-file\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372693    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-binary-copy\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372704    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-system-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372729    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wf8l\" (UniqueName: \"kubernetes.io/projected/542a6603-94a1-4a3b-afc1-589dbbdb7e27-kube-api-access-9wf8l\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-kubelet-config\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372762    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4225a046-a4d4-498f-a7e6-587bd1fd6017-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372781    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-dbus\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372805    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-device-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372828    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc515b02-bb79-4f8d-9122-ca6043180ae8-tmp-dir\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-netns\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-multus\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372903    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372926    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542a6603-94a1-4a3b-afc1-589dbbdb7e27-host-slash\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372946    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-hostroot\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.372949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-cnibin\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-tuning-conf-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373006    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-cnibin\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373062    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0f71116-392c-4b1f-888c-1995314b1cd2-cni-binary-copy\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373106    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-socket-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373135    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-var-lib-cni-multus\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373138    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-system-cni-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373174    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-device-dir\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.373191    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373241    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4225a046-a4d4-498f-a7e6-587bd1fd6017-tuning-conf-dir\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.373248    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:12.873232533 +0000 UTC m=+3.082040901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373292    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542a6603-94a1-4a3b-afc1-589dbbdb7e27-host-slash\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373306    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/542a6603-94a1-4a3b-afc1-589dbbdb7e27-iptables-alerter-script\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373366    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-host-run-netns\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.374611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373394    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc515b02-bb79-4f8d-9122-ca6043180ae8-tmp-dir\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373393    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-dbus\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373482    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-socket-dir-parent\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373490    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0f71116-392c-4b1f-888c-1995314b1cd2-multus-conf-dir\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373548    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-kubelet-config\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.373655    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc515b02-bb79-4f8d-9122-ca6043180ae8-hosts-file\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.375129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.374259    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/523eaa34-2dd5-4ec3-b9c1-4207ee722ef7-agent-certs\") pod \"konnectivity-agent-d8xgh\" (UID: \"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7\") " pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.390114 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.390073    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfbpp\" (UniqueName: \"kubernetes.io/projected/fc515b02-bb79-4f8d-9122-ca6043180ae8-kube-api-access-mfbpp\") pod \"node-resolver-b9cph\" (UID: \"fc515b02-bb79-4f8d-9122-ca6043180ae8\") " pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.390272 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.390167    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tstbr\" (UniqueName: \"kubernetes.io/projected/f6802753-6849-4cde-8ffa-e3a7cdcf64ae-kube-api-access-tstbr\") pod \"aws-ebs-csi-driver-node-j26nq\" (UID: \"f6802753-6849-4cde-8ffa-e3a7cdcf64ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.390272 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.390261    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wf8l\" (UniqueName: \"kubernetes.io/projected/542a6603-94a1-4a3b-afc1-589dbbdb7e27-kube-api-access-9wf8l\") pod \"iptables-alerter-2wrmd\" (UID: \"542a6603-94a1-4a3b-afc1-589dbbdb7e27\") " pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.390362 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.390322    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgzv4\" (UniqueName: \"kubernetes.io/projected/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-kube-api-access-hgzv4\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.393069 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.393047    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:12.393164 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.393075    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:12.393164 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.393089    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:12.393164 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.393154    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:12.893136724 +0000 UTC m=+3.101945106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:12.399867 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.399832    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhhmb\" (UniqueName: \"kubernetes.io/projected/4225a046-a4d4-498f-a7e6-587bd1fd6017-kube-api-access-zhhmb\") pod \"multus-additional-cni-plugins-522sb\" (UID: \"4225a046-a4d4-498f-a7e6-587bd1fd6017\") " pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.416906 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.416832    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-522xn\" (UniqueName: \"kubernetes.io/projected/f0f71116-392c-4b1f-888c-1995314b1cd2-kube-api-access-522xn\") pod \"multus-ggh7s\" (UID: \"f0f71116-392c-4b1f-888c-1995314b1cd2\") " pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.468492 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.468464    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:24:12.468492 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.468498    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:12.478661 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.478633    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac49d6c_415f_4ab6_9cfe_2809aac3423e.slice/crio-873a619a6aa1ea1c54a3f6a30d531c4f7306f28e20e2da8b3b4d714fa2cf7cf6 WatchSource:0}: Error finding container 873a619a6aa1ea1c54a3f6a30d531c4f7306f28e20e2da8b3b4d714fa2cf7cf6: Status 404 returned error can't find the container with id 873a619a6aa1ea1c54a3f6a30d531c4f7306f28e20e2da8b3b4d714fa2cf7cf6
Jul 06 14:24:12.480676 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.480655    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xd45t"
Jul 06 14:24:12.489452 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.489426    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23730560_a85b_48ec_b576_1b8164de769e.slice/crio-ae9d7061342372d10387ed900d2c0c342f4e11e6c01b445547cdae40401bc9ce WatchSource:0}: Error finding container ae9d7061342372d10387ed900d2c0c342f4e11e6c01b445547cdae40401bc9ce: Status 404 returned error can't find the container with id ae9d7061342372d10387ed900d2c0c342f4e11e6c01b445547cdae40401bc9ce
Jul 06 14:24:12.490442 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.490382    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6vdx"
Jul 06 14:24:12.497133 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.496943    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ggh7s"
Jul 06 14:24:12.500203 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.500170    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ed3cfe_e58e_4b15_ad73_2c821535d27d.slice/crio-89361c8ff25b16c72510a18e22b9ecb32c8e12fcdd3fc58d0eb2643cab8263cf WatchSource:0}: Error finding container 89361c8ff25b16c72510a18e22b9ecb32c8e12fcdd3fc58d0eb2643cab8263cf: Status 404 returned error can't find the container with id 89361c8ff25b16c72510a18e22b9ecb32c8e12fcdd3fc58d0eb2643cab8263cf
Jul 06 14:24:12.505754 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.505727    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f71116_392c_4b1f_888c_1995314b1cd2.slice/crio-3d7728c983f4a181d96f1e14831429921a9c4f60d68695631db29c519d603294 WatchSource:0}: Error finding container 3d7728c983f4a181d96f1e14831429921a9c4f60d68695631db29c519d603294: Status 404 returned error can't find the container with id 3d7728c983f4a181d96f1e14831429921a9c4f60d68695631db29c519d603294
Jul 06 14:24:12.506593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.506577    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2wrmd"
Jul 06 14:24:12.516063 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.516032    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542a6603_94a1_4a3b_afc1_589dbbdb7e27.slice/crio-67bb8fcc2532ad793376768607ec2adb9112a45b98cf73bf35a24da3e67103f9 WatchSource:0}: Error finding container 67bb8fcc2532ad793376768607ec2adb9112a45b98cf73bf35a24da3e67103f9: Status 404 returned error can't find the container with id 67bb8fcc2532ad793376768607ec2adb9112a45b98cf73bf35a24da3e67103f9
Jul 06 14:24:12.522912 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.522889    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-522sb"
Jul 06 14:24:12.529664 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.529639    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:12.531817 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.531785    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4225a046_a4d4_498f_a7e6_587bd1fd6017.slice/crio-9ae62e5f23c9648322fac74bf659c0f99773af4e4803d7857178087e2168bf28 WatchSource:0}: Error finding container 9ae62e5f23c9648322fac74bf659c0f99773af4e4803d7857178087e2168bf28: Status 404 returned error can't find the container with id 9ae62e5f23c9648322fac74bf659c0f99773af4e4803d7857178087e2168bf28
Jul 06 14:24:12.537610 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.537587    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq"
Jul 06 14:24:12.541666 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.541639    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523eaa34_2dd5_4ec3_b9c1_4207ee722ef7.slice/crio-d09fd96636dcaf729e54221a195d050a9669ede43b58316ab7921e71d5e29bab WatchSource:0}: Error finding container d09fd96636dcaf729e54221a195d050a9669ede43b58316ab7921e71d5e29bab: Status 404 returned error can't find the container with id d09fd96636dcaf729e54221a195d050a9669ede43b58316ab7921e71d5e29bab
Jul 06 14:24:12.544481 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.544435    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b9cph"
Jul 06 14:24:12.549324 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.549296    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6802753_6849_4cde_8ffa_e3a7cdcf64ae.slice/crio-a25d50ad0e9ed7c6654ec923947ae2b072b1a8049f96003d5bd245144637519a WatchSource:0}: Error finding container a25d50ad0e9ed7c6654ec923947ae2b072b1a8049f96003d5bd245144637519a: Status 404 returned error can't find the container with id a25d50ad0e9ed7c6654ec923947ae2b072b1a8049f96003d5bd245144637519a
Jul 06 14:24:12.557862 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:12.557832    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc515b02_bb79_4f8d_9122_ca6043180ae8.slice/crio-26857cf7917c01793330541fd1e4666a16efe782d9f193a900c0dcb74231d057 WatchSource:0}: Error finding container 26857cf7917c01793330541fd1e4666a16efe782d9f193a900c0dcb74231d057: Status 404 returned error can't find the container with id 26857cf7917c01793330541fd1e4666a16efe782d9f193a900c0dcb74231d057
Jul 06 14:24:12.875937 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.875898    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:12.876121 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.875979    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:12.876121 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.876078    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:12.876121 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.876094    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:12.876249 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.876155    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:13.876135208 +0000 UTC m=+4.084943578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:12.876249 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.876175    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:13.876165303 +0000 UTC m=+4.084973670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:12.976624 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:12.976596    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:12.976785 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.976769    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:12.976850 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.976789    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:12.976850 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.976800    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:12.976933 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:12.976863    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:13.976843958 +0000 UTC m=+4.185652331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:13.286617 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.286539    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:13.287072 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.286664    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:13.287072 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.286720    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:13.287072 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.286835    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:13.294886 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.294848    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:19:12 +0000 UTC" deadline="2028-04-13 20:08:20.952590556 +0000 UTC"
Jul 06 14:24:13.294886 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.294883    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15533h44m7.657712419s"
Jul 06 14:24:13.295144 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.294969    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b9cph" event={"ID":"fc515b02-bb79-4f8d-9122-ca6043180ae8","Type":"ContainerStarted","Data":"26857cf7917c01793330541fd1e4666a16efe782d9f193a900c0dcb74231d057"}
Jul 06 14:24:13.296307 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.296278    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2wrmd" event={"ID":"542a6603-94a1-4a3b-afc1-589dbbdb7e27","Type":"ContainerStarted","Data":"67bb8fcc2532ad793376768607ec2adb9112a45b98cf73bf35a24da3e67103f9"}
Jul 06 14:24:13.297473 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.297452    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ggh7s" event={"ID":"f0f71116-392c-4b1f-888c-1995314b1cd2","Type":"ContainerStarted","Data":"3d7728c983f4a181d96f1e14831429921a9c4f60d68695631db29c519d603294"}
Jul 06 14:24:13.298662 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.298637    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6vdx" event={"ID":"c6ed3cfe-e58e-4b15-ad73-2c821535d27d","Type":"ContainerStarted","Data":"89361c8ff25b16c72510a18e22b9ecb32c8e12fcdd3fc58d0eb2643cab8263cf"}
Jul 06 14:24:13.299825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.299805    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"873a619a6aa1ea1c54a3f6a30d531c4f7306f28e20e2da8b3b4d714fa2cf7cf6"}
Jul 06 14:24:13.301006 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.300967    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq" event={"ID":"f6802753-6849-4cde-8ffa-e3a7cdcf64ae","Type":"ContainerStarted","Data":"a25d50ad0e9ed7c6654ec923947ae2b072b1a8049f96003d5bd245144637519a"}
Jul 06 14:24:13.302125 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.302100    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d8xgh" event={"ID":"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7","Type":"ContainerStarted","Data":"d09fd96636dcaf729e54221a195d050a9669ede43b58316ab7921e71d5e29bab"}
Jul 06 14:24:13.303126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.303104    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerStarted","Data":"9ae62e5f23c9648322fac74bf659c0f99773af4e4803d7857178087e2168bf28"}
Jul 06 14:24:13.304296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.304275    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xd45t" event={"ID":"23730560-a85b-48ec-b576-1b8164de769e","Type":"ContainerStarted","Data":"ae9d7061342372d10387ed900d2c0c342f4e11e6c01b445547cdae40401bc9ce"}
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.882783    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.882964    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.883107    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.883182    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:15.883151741 +0000 UTC m=+6.091960117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.883229    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:13.883308 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.883280    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:15.883264747 +0000 UTC m=+6.092073113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:13.983767 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:13.983721    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:13.984033 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.983904    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:13.984033 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.983925    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:13.984033 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.983937    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:13.984033 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:13.984022    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:15.983986686 +0000 UTC m=+6.192795056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:14.287960 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:14.287912    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:14.288414 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:14.288101    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:14.295796 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:14.295699    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:19:12 +0000 UTC" deadline="2028-03-23 15:44:24.524990736 +0000 UTC"
Jul 06 14:24:14.295796 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:14.295758    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15025h20m10.2292363s"
Jul 06 14:24:14.338215 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:14.338144    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal" event={"ID":"7ee29cf4a0b9a74c02a59c261bf9c8fd","Type":"ContainerStarted","Data":"c3b2c9a544754180f62fcc1bfc5f12b1b793e537f63ad563e78632356a3a5157"}
Jul 06 14:24:15.287052 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.287011    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:15.287226 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.287199    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:15.287697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.287667    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:15.287805 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.287769    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:15.353660 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.353623    2570 generic.go:356] "Generic (PLEG): container finished" podID="90b8b6363e2d727a9190ed846d2b0458" containerID="9082e20f326f8c87ca10e172bb42698b4ca9bf2f905bfc2a6bd04d9df6651283" exitCode=0
Jul 06 14:24:15.354616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.354573    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal" event={"ID":"90b8b6363e2d727a9190ed846d2b0458","Type":"ContainerDied","Data":"9082e20f326f8c87ca10e172bb42698b4ca9bf2f905bfc2a6bd04d9df6651283"}
Jul 06 14:24:15.370972 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.370911    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-204.ec2.internal" podStartSLOduration=4.370889116 podStartE2EDuration="4.370889116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:14.356738858 +0000 UTC m=+4.565547261" watchObservedRunningTime="2026-07-06 14:24:15.370889116 +0000 UTC m=+5.579697502"
Jul 06 14:24:15.901090 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.901049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:15.901289 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:15.901136    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:15.901364 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.901296    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:15.901364 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.901360    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:19.901340798 +0000 UTC m=+10.110149165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:15.901769 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.901749    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:15.901869 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:15.901805    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:19.901790059 +0000 UTC m=+10.110598429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:16.002136 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:16.002098    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:16.002323 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:16.002293    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:16.002323 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:16.002314    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:16.002440 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:16.002326    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:16.002440 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:16.002388    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:20.002369799 +0000 UTC m=+10.211178190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:16.289677 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:16.289117    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:16.289677 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:16.289262    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:17.287346 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:17.286875    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:17.287346 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:17.287035    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:17.287346 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:17.287247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:17.287346 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:17.287336    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:18.288851 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:18.288796    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:18.289337 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:18.288952    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:19.286449 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:19.286382    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:19.286633 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.286533    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:19.286633 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:19.286546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:19.286755 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.286665    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:19.931269 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:19.931232    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:19.931739 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:19.931284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:19.931739 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.931381    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:19.931739 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.931453    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:27.931434612 +0000 UTC m=+18.140242988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:19.931739 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.931381    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:19.931739 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:19.931509    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:27.931496192 +0000 UTC m=+18.140304555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:20.032235 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:20.032194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:20.032435 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:20.032384    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:20.032435 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:20.032409    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:20.032435 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:20.032423    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:20.032617 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:20.032493    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:28.032472799 +0000 UTC m=+18.241281178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:20.288078 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:20.287962    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:20.288922 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:20.288490    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:21.287068 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:21.287034    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:21.287497 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:21.287039    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:21.287497 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:21.287191    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:21.287497 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:21.287268    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:22.287340 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:22.287304    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:22.287761 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:22.287460    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:23.286528 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:23.286497    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:23.286613 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:23.286536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:23.286652 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:23.286607    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:23.286724 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:23.286708    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:24.286674 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:24.286594    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:24.287809 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:24.286738    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:25.286804 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:25.286768    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:25.287289 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:25.286907    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:25.287289 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:25.286984    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:25.287289 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:25.287118    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:26.287098 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:26.287057    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:26.287504 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:26.287195    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:27.287158 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:27.287125    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:27.287158 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:27.287146    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:27.287724 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.287254    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:27.287724 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.287388    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:27.990663 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:27.990620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:27.990915 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:27.990707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:27.990915 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.990803    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:27.990915 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.990818    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:27.990915 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.990882    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:24:43.990860856 +0000 UTC m=+34.199669224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:27.990915 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:27.990899    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:24:43.990892298 +0000 UTC m=+34.199700661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:28.091380 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:28.091345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:28.091562 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:28.091498    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:28.091562 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:28.091521    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:28.091562 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:28.091536    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:28.091696 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:28.091599    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:44.091583642 +0000 UTC m=+34.300392006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:28.287048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:28.286954    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:28.287187 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:28.287138    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:29.287037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:29.286980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:29.287216 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:29.286980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:29.287216 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:29.287111    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:29.287216 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:29.287169    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:30.287708 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:30.287677    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:30.288204 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:30.287798    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:31.286636 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:31.286595    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:31.286821 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:31.286735    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:31.286886 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:31.286814    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:31.286946 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:31.286915    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:32.287287 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.286960    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:32.287924 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:32.287402    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:32.388211 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.388128    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ggh7s" event={"ID":"f0f71116-392c-4b1f-888c-1995314b1cd2","Type":"ContainerStarted","Data":"492bf2b41b17408e8617cdf7ef393cedb66d761793c480cdb0acd68b72ff1499"}
Jul 06 14:24:32.389683 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.389656    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6vdx" event={"ID":"c6ed3cfe-e58e-4b15-ad73-2c821535d27d","Type":"ContainerStarted","Data":"4a59bdccbce772e2b22353a7b7e17e5e89aee5d45bbd7629dce085c89a3ef063"}
Jul 06 14:24:32.392916 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.392886    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"295924f753ea92010579a233288ace2ef3d2a39e10da474917dc44a70a005064"}
Jul 06 14:24:32.393073 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.392970    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"d892fc55749081f17553259910430044ff6d1d7b351dc3f835c841f67228e9fd"}
Jul 06 14:24:32.393073 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.393039    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"fae6c3e5485a3faa2e281027fd7a7bccff63f11413e13f3f2c714d3ea900c77a"}
Jul 06 14:24:32.393199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.393091    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"0ee7a9add16e721c391b24dd089f551a5ad03a73f94565a3a0c4f933ace444c6"}
Jul 06 14:24:32.393199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.393140    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"b5bcc5bd4102fb5c97d46c52b9465d2aa9cf6e17d4bb575d58da24d3111d787a"}
Jul 06 14:24:32.393199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.393188    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"f690b0ca75a41f3875f43af6da55f0b448e7960269608909f4f7d8fb427ea089"}
Jul 06 14:24:32.394915 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.394893    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal" event={"ID":"90b8b6363e2d727a9190ed846d2b0458","Type":"ContainerStarted","Data":"fe1d2f8ca77a3e0014abb2cf5296d7e31a9b229c5b8f32f1e4d680f30f2244df"}
Jul 06 14:24:32.396256 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.396238    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq" event={"ID":"f6802753-6849-4cde-8ffa-e3a7cdcf64ae","Type":"ContainerStarted","Data":"a756dfc742bc04a6e4605ea1e897adcd7c59af43b4b022d0cf555e8037ceb84b"}
Jul 06 14:24:32.397579 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.397557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d8xgh" event={"ID":"523eaa34-2dd5-4ec3-b9c1-4207ee722ef7","Type":"ContainerStarted","Data":"97d21ac8b53ead756e6fb8a76fa7ed56c5e156ffd3b61e471dd59008d038d59a"}
Jul 06 14:24:32.399097 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.399076    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="ced26eaea31d7616fe30227f8c782ccc9e79e2450fce0463a29a91db38dfcb71" exitCode=0
Jul 06 14:24:32.399202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.399138    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"ced26eaea31d7616fe30227f8c782ccc9e79e2450fce0463a29a91db38dfcb71"}
Jul 06 14:24:32.400642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.400613    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xd45t" event={"ID":"23730560-a85b-48ec-b576-1b8164de769e","Type":"ContainerStarted","Data":"60a72368febed75e24e7d1d3238a4dbb09b6d93ca63951000449a13fd0c352b8"}
Jul 06 14:24:32.402313 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.402115    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b9cph" event={"ID":"fc515b02-bb79-4f8d-9122-ca6043180ae8","Type":"ContainerStarted","Data":"4b62a20d68491e4f62cfea926c193196100b6e44ebdee0887c696b56364a0c5d"}
Jul 06 14:24:32.406054 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.405973    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ggh7s" podStartSLOduration=3.248696488 podStartE2EDuration="22.405962179s" totalImagesPullingTime="19.157265691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.405559089 +0000 UTC m=+22.614367472" watchObservedRunningTime="2026-07-06 14:24:32.405962179 +0000 UTC m=+22.614770564"
Jul 06 14:24:32.420559 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.420520    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-204.ec2.internal" podStartSLOduration=21.420508178 podStartE2EDuration="21.420508178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.420485435 +0000 UTC m=+22.629293821" watchObservedRunningTime="2026-07-06 14:24:32.420508178 +0000 UTC m=+22.629316562"
Jul 06 14:24:32.438181 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.438139    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xd45t" podStartSLOduration=3.329126427 podStartE2EDuration="22.438124548s" totalImagesPullingTime="19.108998121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.437675507 +0000 UTC m=+22.646483893" watchObservedRunningTime="2026-07-06 14:24:32.438124548 +0000 UTC m=+22.646932933"
Jul 06 14:24:32.488814 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.488757    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d8xgh" podStartSLOduration=3.432450174 podStartE2EDuration="22.488743175s" totalImagesPullingTime="19.056293001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.488405344 +0000 UTC m=+22.697213730" watchObservedRunningTime="2026-07-06 14:24:32.488743175 +0000 UTC m=+22.697551561"
Jul 06 14:24:32.505119 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.505067    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v6vdx" podStartSLOduration=3.406599191 podStartE2EDuration="22.50505377s" totalImagesPullingTime="19.098454579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.504497973 +0000 UTC m=+22.713306359" watchObservedRunningTime="2026-07-06 14:24:32.50505377 +0000 UTC m=+22.713862156"
Jul 06 14:24:32.521528 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:32.521481    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b9cph" podStartSLOduration=3.458087489 podStartE2EDuration="22.521469891s" totalImagesPullingTime="19.063382402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:32.521289104 +0000 UTC m=+22.730097490" watchObservedRunningTime="2026-07-06 14:24:32.521469891 +0000 UTC m=+22.730278276"
Jul 06 14:24:33.286682 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:33.286608    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:33.287014 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:33.286763    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:33.287014 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:33.286784    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:33.287014 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:33.286862    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:33.405588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:33.405562    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2wrmd" event={"ID":"542a6603-94a1-4a3b-afc1-589dbbdb7e27","Type":"ContainerStarted","Data":"2758627ae23f3d52f44a86386236c363d3060871b6faeb583389e005146f2466"}
Jul 06 14:24:33.434743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:33.434688    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2wrmd" podStartSLOduration=4.352027104 podStartE2EDuration="23.434667556s" totalImagesPullingTime="19.082640452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:33.434416975 +0000 UTC m=+23.643225379" watchObservedRunningTime="2026-07-06 14:24:33.434667556 +0000 UTC m=+23.643475943"
Jul 06 14:24:33.570333 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:33.570306    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 14:24:34.212573 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.212450    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:24:33.570326085Z","UUID":"0db40d1d-113c-455d-b478-c6f4912a7efa","Handler":null,"Name":"","Endpoint":""}
Jul 06 14:24:34.214301 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.214276    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 14:24:34.214451 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.214319    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 14:24:34.217802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.217770    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:34.218345 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.218323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:34.287211 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.287179    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:34.287390 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:34.287318    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:34.410371 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:34.410335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq" event={"ID":"f6802753-6849-4cde-8ffa-e3a7cdcf64ae","Type":"ContainerStarted","Data":"313025b6c7951800caffea44966eb228fc20c0f16595f808f27ed26db6a2aa73"}
Jul 06 14:24:35.287543 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.287291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:35.287716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.287344    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:35.287716 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:35.287583    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:35.287716 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:35.287639    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:35.414665 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.414626    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq" event={"ID":"f6802753-6849-4cde-8ffa-e3a7cdcf64ae","Type":"ContainerStarted","Data":"bf7ba6a7822a0b63953e6a3d8f070f93f83ec6a747ba899fbdbd5bc7a8cc1206"}
Jul 06 14:24:35.417818 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.417790    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"9774349b16854103af9689e40bb7ea7c1cf1c943267c1080f01a5bed8e431d31"}
Jul 06 14:24:35.417818 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.417818    2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 14:24:35.433752 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:35.433702    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j26nq" podStartSLOduration=3.502987133 podStartE2EDuration="25.433681286s" totalImagesPullingTime="21.930694153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:35.433551992 +0000 UTC m=+25.642360410" watchObservedRunningTime="2026-07-06 14:24:35.433681286 +0000 UTC m=+25.642489675"
Jul 06 14:24:36.033750 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:36.033710    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:36.034450 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:36.034406    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d8xgh"
Jul 06 14:24:36.287091 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:36.287022    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:36.287230 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:36.287134    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:37.286581 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:37.286352    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:37.287252 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:37.286409    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:37.287252 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:37.286699    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:37.287252 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:37.286778    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:37.426694 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:37.426653    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" event={"ID":"7ac49d6c-415f-4ab6-9cfe-2809aac3423e","Type":"ContainerStarted","Data":"1d07ec8a73672474e7327f7edb50b50382d7100aaa07631d40980210dc249ecc"}
Jul 06 14:24:37.463732 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:37.463678    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx" podStartSLOduration=8.249679592 podStartE2EDuration="27.463660782s" totalImagesPullingTime="19.21398119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:37.463144481 +0000 UTC m=+27.671952904" watchObservedRunningTime="2026-07-06 14:24:37.463660782 +0000 UTC m=+27.672469166"
Jul 06 14:24:38.286465 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.286393    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:38.286639 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:38.286539    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:38.429489 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.429422    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:38.429634 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.429506    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:38.429634 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.429554    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:38.448373 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.448340    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:38.448633 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.448602    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:24:38.605582 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.605550    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hf2kl"]
Jul 06 14:24:38.605763 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.605727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:38.605840 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:38.605817    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:38.608366 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.608339    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wlxc7"]
Jul 06 14:24:38.608484 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.608454    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:38.608567 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:38.608546    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:38.611370 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.611346    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-smh7l"]
Jul 06 14:24:38.611484 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:38.611471    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:38.611604 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:38.611577    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:39.432595 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:39.432559    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="f13dba280353434ba1b35220aa0768199ba74cf558bba5c6904068024e270824" exitCode=0
Jul 06 14:24:39.432937 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:39.432599    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"f13dba280353434ba1b35220aa0768199ba74cf558bba5c6904068024e270824"}
Jul 06 14:24:40.288008 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:40.287959    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:40.288199 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:40.288122    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:40.288199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:40.288153    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:40.288339 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:40.288255    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:40.288339 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:40.288283    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:40.288339 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:40.288326    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:41.439742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:41.439519    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="3e0a2af3d0b77b57f65c13fd86d14fec955e18febc15159d201a396a21d1e0ff" exitCode=0
Jul 06 14:24:41.440293 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:41.439604    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"3e0a2af3d0b77b57f65c13fd86d14fec955e18febc15159d201a396a21d1e0ff"}
Jul 06 14:24:42.287215 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:42.287180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:42.287389 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:42.287180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:42.287389 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:42.287303    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:42.287389 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:42.287180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:42.287389 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:42.287364    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:42.287609 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:42.287417    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:43.447296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:43.447242    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="38aefa22aecd3b90adb0f00278041f01aeae0ba51ea56e8c7b09e42529d99154" exitCode=0
Jul 06 14:24:43.447868 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:43.447306    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"38aefa22aecd3b90adb0f00278041f01aeae0ba51ea56e8c7b09e42529d99154"}
Jul 06 14:24:44.009717 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.009683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:44.010009 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.009728    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:44.010009 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.009827    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:44.010009 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.009837    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:44.010009 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.009877    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.009861443 +0000 UTC m=+66.218669807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:24:44.010009 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.009906    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret podName:c3a36fc8-b9bd-47d1-963e-46d65a8867dc nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.009888163 +0000 UTC m=+66.218696541 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret") pod "global-pull-secret-syncer-wlxc7" (UID: "c3a36fc8-b9bd-47d1-963e-46d65a8867dc") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:24:44.110753 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.110717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:44.110931 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.110861    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:24:44.110931 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.110880    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:24:44.110931 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.110892    2570 projected.go:196] Error preparing data for projected volume kube-api-access-b2jxg for pod openshift-network-diagnostics/network-check-target-hf2kl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:44.111122 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.111006    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg podName:8fd36dbf-f40d-43a5-918e-130ff5128ac4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.110973376 +0000 UTC m=+66.319781744 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2jxg" (UniqueName: "kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg") pod "network-check-target-hf2kl" (UID: "8fd36dbf-f40d-43a5-918e-130ff5128ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:24:44.290133 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.290060    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:44.290296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.290060    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:44.290296 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.290156    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-smh7l" podUID="e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b"
Jul 06 14:24:44.290296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.290060    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:44.290296 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.290240    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wlxc7" podUID="c3a36fc8-b9bd-47d1-963e-46d65a8867dc"
Jul 06 14:24:44.290434 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.290329    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hf2kl" podUID="8fd36dbf-f40d-43a5-918e-130ff5128ac4"
Jul 06 14:24:44.616894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.616860    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-204.ec2.internal" event="NodeReady"
Jul 06 14:24:44.617422 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.617037    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 14:24:44.672252 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.672214    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:24:44.674249 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.674219    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.679663 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.679638    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 14:24:44.679791 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.679675    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 14:24:44.679877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.679820    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 14:24:44.691491 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.691466    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9dzg8\""
Jul 06 14:24:44.700415 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.700390    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 14:24:44.702706 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.702682    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vls8r"]
Jul 06 14:24:44.704572 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.704553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.707372 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.707352    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z7n64"]
Jul 06 14:24:44.708150 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.708123    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 14:24:44.708273 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.708166    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 14:24:44.708273 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.708208    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s54bc\""
Jul 06 14:24:44.708971 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.708955    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.711616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.711598    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 14:24:44.712143 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.712118    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 14:24:44.712239 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.712196    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 14:24:44.713025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.713010    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqgrr\""
Jul 06 14:24:44.718947 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.718924    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vls8r"]
Jul 06 14:24:44.728264 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.728129    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z7n64"]
Jul 06 14:24:44.745627 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.745525    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:24:44.815324 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815289    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad56e44f-e089-427b-9532-00d941fa3b80-config-volume\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.815324 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815321    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.815545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815349    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad56e44f-e089-427b-9532-00d941fa3b80-tmp-dir\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.815545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815380    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b246x\" (UniqueName: \"kubernetes.io/projected/ad56e44f-e089-427b-9532-00d941fa3b80-kube-api-access-b246x\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.815545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815479    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815509    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815727 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815564    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815727 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815727 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815614    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh7pb\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815727 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815672    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815728    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815775    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-cert\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.815911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.815911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.815830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6bp2\" (UniqueName: \"kubernetes.io/projected/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-kube-api-access-m6bp2\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.916898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.916898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916834    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.916898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh7pb\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.916898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916885    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916913    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-cert\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916960    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.916977    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6bp2\" (UniqueName: \"kubernetes.io/projected/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-kube-api-access-m6bp2\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad56e44f-e089-427b-9532-00d941fa3b80-config-volume\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917064    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad56e44f-e089-427b-9532-00d941fa3b80-tmp-dir\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b246x\" (UniqueName: \"kubernetes.io/projected/ad56e44f-e089-427b-9532-00d941fa3b80-kube-api-access-b246x\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917128    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917150    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.917263    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:24:44.917424 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.917328    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:45.417308913 +0000 UTC m=+35.626117281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:24:44.917978 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.917654    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:24:44.917978 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.917672    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:24:44.917978 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:44.917728    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:45.417712499 +0000 UTC m=+35.626520877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:24:44.917978 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.917748    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad56e44f-e089-427b-9532-00d941fa3b80-config-volume\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.918182 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.918024    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad56e44f-e089-427b-9532-00d941fa3b80-tmp-dir\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.918565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.918529    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.918660 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.918537    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.919279 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.919250    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.921743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.921717    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.921743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.921735    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-cert\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.921912 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.921865    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.933152 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.933125    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:44.939151 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.939127    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b246x\" (UniqueName: \"kubernetes.io/projected/ad56e44f-e089-427b-9532-00d941fa3b80-kube-api-access-b246x\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:44.939151 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.939143    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6bp2\" (UniqueName: \"kubernetes.io/projected/29f5a40d-93f2-434a-8da5-624bdbd9b4cc-kube-api-access-m6bp2\") pod \"ingress-canary-z7n64\" (UID: \"29f5a40d-93f2-434a-8da5-624bdbd9b4cc\") " pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:44.940212 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:44.940186    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh7pb\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:45.020867 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.020828    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z7n64"
Jul 06 14:24:45.150851 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.150588    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:45.200207 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.200086    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z7n64"]
Jul 06 14:24:45.203927 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:24:45.203895    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f5a40d_93f2_434a_8da5_624bdbd9b4cc.slice/crio-66120e816c5a77755d99f4155f4cee3ea9279732386bc4eca939d618821a82d2 WatchSource:0}: Error finding container 66120e816c5a77755d99f4155f4cee3ea9279732386bc4eca939d618821a82d2: Status 404 returned error can't find the container with id 66120e816c5a77755d99f4155f4cee3ea9279732386bc4eca939d618821a82d2
Jul 06 14:24:45.265216 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.265181    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:45.421320 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.421286    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:45.421523 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.421391    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:45.421523 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:45.421461    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:24:45.421523 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:45.421517    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:24:45.421692 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:45.421532    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:24:45.421692 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:45.421539    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:46.421522874 +0000 UTC m=+36.630331243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:24:45.421806 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:45.421763    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:46.421665419 +0000 UTC m=+36.630473796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:24:45.453752 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:45.453669    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z7n64" event={"ID":"29f5a40d-93f2-434a-8da5-624bdbd9b4cc","Type":"ContainerStarted","Data":"66120e816c5a77755d99f4155f4cee3ea9279732386bc4eca939d618821a82d2"}
Jul 06 14:24:46.286553 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.286514    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:24:46.287051 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.286514    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:24:46.287051 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.286521    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:24:46.290958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.290865    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 14:24:46.290958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.290882    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 14:24:46.290958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.290920    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fzm8z\""
Jul 06 14:24:46.291217 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.291007    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-689l7\""
Jul 06 14:24:46.291217 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.290865    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 14:24:46.291217 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.290866    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 14:24:46.430032 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.429980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:46.430225 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:46.430069    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:46.430225 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:46.430102    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:24:46.430225 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:46.430117    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:24:46.430225 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:46.430175    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:48.430157429 +0000 UTC m=+38.638965814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:24:46.430655 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:46.430630    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:24:46.430771 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:46.430702    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:48.430681518 +0000 UTC m=+38.639489895 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:24:48.448724 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:48.448599    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:48.448724 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:48.448665    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:48.449328 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:48.448830    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:24:48.449328 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:48.448894    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:52.448877212 +0000 UTC m=+42.657685576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:24:48.449443 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:48.449358    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:24:48.449443 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:48.449377    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:24:48.449443 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:48.449426    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:24:52.449409379 +0000 UTC m=+42.658217749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:24:49.202009 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:49.201954    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:49.268619 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:49.268585    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:50.008324 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.008295    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:50.092575 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.092547    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:50.262704 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:50.262674    2570 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4225a046_a4d4_498f_a7e6_587bd1fd6017.slice/crio-conmon-aa8066be29f5f455b82720238082a2287dc5aafe88dc8178d228153443d8b612.scope\": RecentStats: unable to find data in memory cache]"
Jul 06 14:24:50.469260 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.469229    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="aa8066be29f5f455b82720238082a2287dc5aafe88dc8178d228153443d8b612" exitCode=0
Jul 06 14:24:50.469440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.469303    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"aa8066be29f5f455b82720238082a2287dc5aafe88dc8178d228153443d8b612"}
Jul 06 14:24:50.470667 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.470633    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z7n64" event={"ID":"29f5a40d-93f2-434a-8da5-624bdbd9b4cc","Type":"ContainerStarted","Data":"a71bed9a7526a5686b2ebb264b868db71665c637289c9dcade4d14baf721fb17"}
Jul 06 14:24:50.507957 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:50.507900    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z7n64" podStartSLOduration=1.707684763 podStartE2EDuration="6.507877311s" totalImagesPullingTime="4.800192548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:50.507723954 +0000 UTC m=+40.716532370" watchObservedRunningTime="2026-07-06 14:24:50.507877311 +0000 UTC m=+40.716685699"
Jul 06 14:24:51.478908 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:51.478869    2570 generic.go:356] "Generic (PLEG): container finished" podID="4225a046-a4d4-498f-a7e6-587bd1fd6017" containerID="68dbe55592739ccedb28cd8e803743054021c5630fe8e0fe32fb4018ece84c6d" exitCode=0
Jul 06 14:24:51.479291 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:51.478952    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerDied","Data":"68dbe55592739ccedb28cd8e803743054021c5630fe8e0fe32fb4018ece84c6d"}
Jul 06 14:24:52.480425 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.480395    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.480440    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:52.480545    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:52.480549    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:52.480564    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:52.480613    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:00.480594446 +0000 UTC m=+50.689402817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:24:52.480887 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:24:52.480630    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:25:00.48062329 +0000 UTC m=+50.689431654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:24:52.484348 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.484320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-522sb" event={"ID":"4225a046-a4d4-498f-a7e6-587bd1fd6017","Type":"ContainerStarted","Data":"afa08f042fcb776fb13de0a249e4589db896f4b4b376d62ab5a825bd5f130b22"}
Jul 06 14:24:52.511217 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.511148    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-522sb" podStartSLOduration=8.08061535 podStartE2EDuration="42.5111325s" totalImagesPullingTime="34.43051715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:52.510865608 +0000 UTC m=+42.719673991" watchObservedRunningTime="2026-07-06 14:24:52.5111325 +0000 UTC m=+42.719940887"
Jul 06 14:24:52.633918 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.633879    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:52.735754 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:52.735646    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:53.222567 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:53.222520    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:24:53.294456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:24:53.294425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log" limit=1024
Jul 06 14:25:00.541292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:00.541252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:00.541306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:00.541424    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:00.541447    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:00.541499    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.541481688 +0000 UTC m=+66.750290079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:00.541430    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:25:00.541784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:00.541563    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:16.541551475 +0000 UTC m=+66.750359839 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:25:10.213588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.213560    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"]
Jul 06 14:25:10.245953 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.245925    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pw8fj"]
Jul 06 14:25:10.246137 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.246081    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.248958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.248924    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 14:25:10.250376 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.250353    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 14:25:10.250488 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.250434    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-978r9\""
Jul 06 14:25:10.250488 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.250460    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 14:25:10.250568 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.250437    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 14:25:10.265787 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.265761    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"]
Jul 06 14:25:10.265787 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.265792    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pw8fj"]
Jul 06 14:25:10.265951 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.265865    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.268596 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.268571    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 06 14:25:10.268738 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.268610    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:25:10.268738 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.268614    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 06 14:25:10.268738 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.268583    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 06 14:25:10.268738 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.268613    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2q2tr\""
Jul 06 14:25:10.274087 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.274067    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 06 14:25:10.296887 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.296858    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"]
Jul 06 14:25:10.322121 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.322096    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"]
Jul 06 14:25:10.322284 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.322216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"
Jul 06 14:25:10.324840 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.324797    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wb4hp\""
Jul 06 14:25:10.339890 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.339856    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"]
Jul 06 14:25:10.340056 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.339964    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.343011 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.342974    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 14:25:10.343124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.342979    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ppldj\""
Jul 06 14:25:10.346468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.346446    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 14:25:10.353707 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.353679    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"]
Jul 06 14:25:10.353829 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.353714    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"]
Jul 06 14:25:10.353829 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.353737    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"]
Jul 06 14:25:10.353829 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.353795    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.356576 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.356554    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 14:25:10.356710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.356613    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 14:25:10.356710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.356632    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2tck9\""
Jul 06 14:25:10.356710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.356667    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 14:25:10.356843 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.356777    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:25:10.394614 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.394582    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-sg7jt"]
Jul 06 14:25:10.413926 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.413895    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-sg7jt"]
Jul 06 14:25:10.414116 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.414018    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.416542 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416520    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 06 14:25:10.416664 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-trusted-ca\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.416730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6b5d\" (UniqueName: \"kubernetes.io/projected/239a5b97-8702-4a74-a4d5-fdad207c183a-kube-api-access-l6b5d\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.416730 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srbbm\" (UniqueName: \"kubernetes.io/projected/0d65dcd3-852a-4605-86f9-fc9307cffad4-kube-api-access-srbbm\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.416838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416777    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.416838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416797    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 14:25:10.416838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-config\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.416838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416835    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 14:25:10.417055 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416864    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239a5b97-8702-4a74-a4d5-fdad207c183a-serving-cert\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.417055 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416944    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0d65dcd3-852a-4605-86f9-fc9307cffad4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.417055 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.416949    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c6h9m\""
Jul 06 14:25:10.417257 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.417242    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 06 14:25:10.422536 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.422517    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 06 14:25:10.454535 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.454510    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxdnx"
Jul 06 14:25:10.518243 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518149    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0d65dcd3-852a-4605-86f9-fc9307cffad4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.518243 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518195    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-trusted-ca\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.518243 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518216    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5sn2\" (UniqueName: \"kubernetes.io/projected/6dda94dd-e4ce-4c50-bb3c-211940025ef0-kube-api-access-f5sn2\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.518243 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srbbm\" (UniqueName: \"kubernetes.io/projected/0d65dcd3-852a-4605-86f9-fc9307cffad4-kube-api-access-srbbm\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518294    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/142f3bd6-dee6-406f-b7d9-3f7da5786619-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518311    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dda94dd-e4ce-4c50-bb3c-211940025ef0-serving-cert\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518333    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a37022f-52f7-46d1-ba09-448059337885-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518357    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518391    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-tmp\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518408    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-snapshots\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518487    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6b5d\" (UniqueName: \"kubernetes.io/projected/239a5b97-8702-4a74-a4d5-fdad207c183a-kube-api-access-l6b5d\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518547    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-config\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.518591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518583    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518618    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77t8\" (UniqueName: \"kubernetes.io/projected/9a37022f-52f7-46d1-ba09-448059337885-kube-api-access-p77t8\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518645    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a37022f-52f7-46d1-ba09-448059337885-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518683    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:10.518738    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518743    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239a5b97-8702-4a74-a4d5-fdad207c183a-serving-cert\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:10.518829    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.018808501 +0000 UTC m=+61.227616864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518879    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8jt\" (UniqueName: \"kubernetes.io/projected/0b4b5a0d-e467-486e-b4be-b7c3caf5b213-kube-api-access-bh8jt\") pod \"network-check-source-989f8b8dc-cnqtk\" (UID: \"0b4b5a0d-e467-486e-b4be-b7c3caf5b213\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518910    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.519044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.518973    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0d65dcd3-852a-4605-86f9-fc9307cffad4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.519418 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.519205    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-config\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.519418 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.519226    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/239a5b97-8702-4a74-a4d5-fdad207c183a-trusted-ca\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.522374 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.522349    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239a5b97-8702-4a74-a4d5-fdad207c183a-serving-cert\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.533933 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.533908    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6b5d\" (UniqueName: \"kubernetes.io/projected/239a5b97-8702-4a74-a4d5-fdad207c183a-kube-api-access-l6b5d\") pod \"console-operator-68b757fb67-pw8fj\" (UID: \"239a5b97-8702-4a74-a4d5-fdad207c183a\") " pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.537469 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.537443    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srbbm\" (UniqueName: \"kubernetes.io/projected/0d65dcd3-852a-4605-86f9-fc9307cffad4-kube-api-access-srbbm\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:10.574303 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.574264    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:10.620049 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620015    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620106    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8jt\" (UniqueName: \"kubernetes.io/projected/0b4b5a0d-e467-486e-b4be-b7c3caf5b213-kube-api-access-bh8jt\") pod \"network-check-source-989f8b8dc-cnqtk\" (UID: \"0b4b5a0d-e467-486e-b4be-b7c3caf5b213\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"
Jul 06 14:25:10.620230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.620230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5sn2\" (UniqueName: \"kubernetes.io/projected/6dda94dd-e4ce-4c50-bb3c-211940025ef0-kube-api-access-f5sn2\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620387 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620311    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/142f3bd6-dee6-406f-b7d9-3f7da5786619-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.620387 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620358    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dda94dd-e4ce-4c50-bb3c-211940025ef0-serving-cert\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620472 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620404    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a37022f-52f7-46d1-ba09-448059337885-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.620472 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620431    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620472 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:10.620452    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:10.620616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-tmp\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620500    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-snapshots\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.620616 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:10.620525    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:11.12050651 +0000 UTC m=+61.329314882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:10.620616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620604    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p77t8\" (UniqueName: \"kubernetes.io/projected/9a37022f-52f7-46d1-ba09-448059337885-kube-api-access-p77t8\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.620825 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.620636    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a37022f-52f7-46d1-ba09-448059337885-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.621139 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621116    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a37022f-52f7-46d1-ba09-448059337885-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.621267 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621195    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.621371 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621339    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dda94dd-e4ce-4c50-bb3c-211940025ef0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.621639 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621552    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-tmp\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.621734 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621717    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/142f3bd6-dee6-406f-b7d9-3f7da5786619-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:10.621892 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.621872    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6dda94dd-e4ce-4c50-bb3c-211940025ef0-snapshots\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.622930 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.622904    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dda94dd-e4ce-4c50-bb3c-211940025ef0-serving-cert\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.623410 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.623393    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a37022f-52f7-46d1-ba09-448059337885-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.632173 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.632139    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77t8\" (UniqueName: \"kubernetes.io/projected/9a37022f-52f7-46d1-ba09-448059337885-kube-api-access-p77t8\") pod \"kube-storage-version-migrator-operator-846bf668f7-98pgc\" (UID: \"9a37022f-52f7-46d1-ba09-448059337885\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.632376 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.632357    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5sn2\" (UniqueName: \"kubernetes.io/projected/6dda94dd-e4ce-4c50-bb3c-211940025ef0-kube-api-access-f5sn2\") pod \"insights-operator-5c847dcfc4-sg7jt\" (UID: \"6dda94dd-e4ce-4c50-bb3c-211940025ef0\") " pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.632503 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.632486    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 14:25:10.643830 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.643804    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 14:25:10.654922 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.654886    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8jt\" (UniqueName: \"kubernetes.io/projected/0b4b5a0d-e467-486e-b4be-b7c3caf5b213-kube-api-access-bh8jt\") pod \"network-check-source-989f8b8dc-cnqtk\" (UID: \"0b4b5a0d-e467-486e-b4be-b7c3caf5b213\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"
Jul 06 14:25:10.662626 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.662595    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"
Jul 06 14:25:10.723808 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.723778    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt"
Jul 06 14:25:10.729264 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.729182    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pw8fj"]
Jul 06 14:25:10.732578 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:10.732543    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239a5b97_8702_4a74_a4d5_fdad207c183a.slice/crio-ab2d4604e18f7134dbf6949e6694891ac11d7de2f2007c34344eb15a6898bba4 WatchSource:0}: Error finding container ab2d4604e18f7134dbf6949e6694891ac11d7de2f2007c34344eb15a6898bba4: Status 404 returned error can't find the container with id ab2d4604e18f7134dbf6949e6694891ac11d7de2f2007c34344eb15a6898bba4
Jul 06 14:25:10.827711 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.827682    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc"]
Jul 06 14:25:10.831811 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:10.831779    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a37022f_52f7_46d1_ba09_448059337885.slice/crio-0c7c52c8d83d6b6b3045255d34766723214404b7c29d267c5d2f9f3e2757fc7a WatchSource:0}: Error finding container 0c7c52c8d83d6b6b3045255d34766723214404b7c29d267c5d2f9f3e2757fc7a: Status 404 returned error can't find the container with id 0c7c52c8d83d6b6b3045255d34766723214404b7c29d267c5d2f9f3e2757fc7a
Jul 06 14:25:10.889784 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.889755    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-sg7jt"]
Jul 06 14:25:10.892837 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:10.892807    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dda94dd_e4ce_4c50_bb3c_211940025ef0.slice/crio-7e94b90348e24881f205970a9c2b21d9405e492b210b43758aa1e5828d0fb025 WatchSource:0}: Error finding container 7e94b90348e24881f205970a9c2b21d9405e492b210b43758aa1e5828d0fb025: Status 404 returned error can't find the container with id 7e94b90348e24881f205970a9c2b21d9405e492b210b43758aa1e5828d0fb025
Jul 06 14:25:10.931748 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:10.931709    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"
Jul 06 14:25:11.024629 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.024548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:11.024777 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:11.024712    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:11.024840 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:11.024788    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:12.024768431 +0000 UTC m=+62.233576795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:11.080241 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.080207    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk"]
Jul 06 14:25:11.083653 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:11.083615    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4b5a0d_e467_486e_b4be_b7c3caf5b213.slice/crio-07a8a48ca2a6f8d5c2e259a4ce5cc1bafcb4c5b634625848148455b1e06a50fa WatchSource:0}: Error finding container 07a8a48ca2a6f8d5c2e259a4ce5cc1bafcb4c5b634625848148455b1e06a50fa: Status 404 returned error can't find the container with id 07a8a48ca2a6f8d5c2e259a4ce5cc1bafcb4c5b634625848148455b1e06a50fa
Jul 06 14:25:11.125671 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.125637    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:11.125827 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:11.125743    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:11.125827 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:11.125802    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:12.125786612 +0000 UTC m=+62.334594976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:11.529029 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.528976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt" event={"ID":"6dda94dd-e4ce-4c50-bb3c-211940025ef0","Type":"ContainerStarted","Data":"7e94b90348e24881f205970a9c2b21d9405e492b210b43758aa1e5828d0fb025"}
Jul 06 14:25:11.530294 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.530268    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk" event={"ID":"0b4b5a0d-e467-486e-b4be-b7c3caf5b213","Type":"ContainerStarted","Data":"07a8a48ca2a6f8d5c2e259a4ce5cc1bafcb4c5b634625848148455b1e06a50fa"}
Jul 06 14:25:11.531759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.531721    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc" event={"ID":"9a37022f-52f7-46d1-ba09-448059337885","Type":"ContainerStarted","Data":"0c7c52c8d83d6b6b3045255d34766723214404b7c29d267c5d2f9f3e2757fc7a"}
Jul 06 14:25:11.532911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:11.532885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" event={"ID":"239a5b97-8702-4a74-a4d5-fdad207c183a","Type":"ContainerStarted","Data":"ab2d4604e18f7134dbf6949e6694891ac11d7de2f2007c34344eb15a6898bba4"}
Jul 06 14:25:12.032830 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:12.032787    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:12.033035 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:12.032976    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:12.033104 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:12.033059    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:14.033037713 +0000 UTC m=+64.241846077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:12.133611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:12.133560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:12.133793 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:12.133761    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:12.133862 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:12.133827    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:14.133807739 +0000 UTC m=+64.342616106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:14.053570 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:14.053529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:14.053981 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:14.053692    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:14.053981 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:14.053770    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.053751362 +0000 UTC m=+68.262559746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:14.154408 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:14.154367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:14.154597 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:14.154537    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:14.154654 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:14.154630    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.15461083 +0000 UTC m=+68.363419206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:15.545712 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.545621    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt" event={"ID":"6dda94dd-e4ce-4c50-bb3c-211940025ef0","Type":"ContainerStarted","Data":"5c5fe2c3de0f8aff30ae65f1127338f7a334df3d12567c693a2ea52752230e51"}
Jul 06 14:25:15.547517 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.547489    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk" event={"ID":"0b4b5a0d-e467-486e-b4be-b7c3caf5b213","Type":"ContainerStarted","Data":"5462a10ffe235f88262a34ea6b2e0857df5d7c931b43f8e298630dfabb7b0254"}
Jul 06 14:25:15.549052 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.549022    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc" event={"ID":"9a37022f-52f7-46d1-ba09-448059337885","Type":"ContainerStarted","Data":"b65997f4c735245c8e31cfda265234aad283b6e08b36dced4d075dae7cc71810"}
Jul 06 14:25:15.550816 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.550795    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/0.log"
Jul 06 14:25:15.550912 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.550835    2570 generic.go:356] "Generic (PLEG): container finished" podID="239a5b97-8702-4a74-a4d5-fdad207c183a" containerID="d3c9df5e6975fda765dd1982d69c2e9e9b9980e48c3391662af57cbbf5650afc" exitCode=255
Jul 06 14:25:15.550912 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.550871    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" event={"ID":"239a5b97-8702-4a74-a4d5-fdad207c183a","Type":"ContainerDied","Data":"d3c9df5e6975fda765dd1982d69c2e9e9b9980e48c3391662af57cbbf5650afc"}
Jul 06 14:25:15.551104 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.551088    2570 scope.go:117] "RemoveContainer" containerID="d3c9df5e6975fda765dd1982d69c2e9e9b9980e48c3391662af57cbbf5650afc"
Jul 06 14:25:15.564639 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.564592    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt" podStartSLOduration=1.2683025780000001 podStartE2EDuration="5.564564156s" totalImagesPullingTime="4.296261578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:15.564137995 +0000 UTC m=+65.772946382" watchObservedRunningTime="2026-07-06 14:25:15.564564156 +0000 UTC m=+65.773372541"
Jul 06 14:25:15.581343 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.581291    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cnqtk" podStartSLOduration=1.467884098 podStartE2EDuration="5.581273027s" totalImagesPullingTime="4.113388929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:15.580191569 +0000 UTC m=+65.788999956" watchObservedRunningTime="2026-07-06 14:25:15.581273027 +0000 UTC m=+65.790081413"
Jul 06 14:25:15.614129 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:15.613818    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc" podStartSLOduration=1.259410479 podStartE2EDuration="5.613796559s" totalImagesPullingTime="4.35438608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:15.612843046 +0000 UTC m=+65.821651424" watchObservedRunningTime="2026-07-06 14:25:15.613796559 +0000 UTC m=+65.822604946"
Jul 06 14:25:16.081966 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.081921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:25:16.082300 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.081983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:25:16.085124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.085103    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 14:25:16.085210 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.085130    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 14:25:16.092478 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.092441    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 14:25:16.092635 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.092528    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs podName:e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b nodeName:}" failed. No retries permitted until 2026-07-06 14:26:20.092506327 +0000 UTC m=+130.301314694 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs") pod "network-metrics-daemon-smh7l" (UID: "e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b") : secret "metrics-daemon-secret" not found
Jul 06 14:25:16.094579 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.094549    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3a36fc8-b9bd-47d1-963e-46d65a8867dc-original-pull-secret\") pod \"global-pull-secret-syncer-wlxc7\" (UID: \"c3a36fc8-b9bd-47d1-963e-46d65a8867dc\") " pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:25:16.183376 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.183324    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:25:16.186048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.186017    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jxg\" (UniqueName: \"kubernetes.io/projected/8fd36dbf-f40d-43a5-918e-130ff5128ac4-kube-api-access-b2jxg\") pod \"network-check-target-hf2kl\" (UID: \"8fd36dbf-f40d-43a5-918e-130ff5128ac4\") " pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:25:16.299899 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.299862    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wlxc7"
Jul 06 14:25:16.310718 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.310696    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fzm8z\""
Jul 06 14:25:16.318555 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.318507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:25:16.475507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.475464    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wlxc7"]
Jul 06 14:25:16.478572 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:16.478542    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a36fc8_b9bd_47d1_963e_46d65a8867dc.slice/crio-c892561bd98c99c012f44c07d2332ed1cf89ff3ab1ba13f6b1398a137f528a3d WatchSource:0}: Error finding container c892561bd98c99c012f44c07d2332ed1cf89ff3ab1ba13f6b1398a137f528a3d: Status 404 returned error can't find the container with id c892561bd98c99c012f44c07d2332ed1cf89ff3ab1ba13f6b1398a137f528a3d
Jul 06 14:25:16.495645 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.495617    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hf2kl"]
Jul 06 14:25:16.499404 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:16.499374    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd36dbf_f40d_43a5_918e_130ff5128ac4.slice/crio-f6f67944d3d8df9711109a7b7b70fc05320f9c3a0c360f7c96baadc0919f3e69 WatchSource:0}: Error finding container f6f67944d3d8df9711109a7b7b70fc05320f9c3a0c360f7c96baadc0919f3e69: Status 404 returned error can't find the container with id f6f67944d3d8df9711109a7b7b70fc05320f9c3a0c360f7c96baadc0919f3e69
Jul 06 14:25:16.556369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.556342    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/1.log"
Jul 06 14:25:16.556872 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.556851    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/0.log"
Jul 06 14:25:16.556944 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.556898    2570 generic.go:356] "Generic (PLEG): container finished" podID="239a5b97-8702-4a74-a4d5-fdad207c183a" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e" exitCode=255
Jul 06 14:25:16.557032 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.557007    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" event={"ID":"239a5b97-8702-4a74-a4d5-fdad207c183a","Type":"ContainerDied","Data":"e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"}
Jul 06 14:25:16.557105 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.557089    2570 scope.go:117] "RemoveContainer" containerID="d3c9df5e6975fda765dd1982d69c2e9e9b9980e48c3391662af57cbbf5650afc"
Jul 06 14:25:16.557379 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.557361    2570 scope.go:117] "RemoveContainer" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"
Jul 06 14:25:16.557576 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.557554    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pw8fj_openshift-console-operator(239a5b97-8702-4a74-a4d5-fdad207c183a)\"" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podUID="239a5b97-8702-4a74-a4d5-fdad207c183a"
Jul 06 14:25:16.558929 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.558896    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hf2kl" event={"ID":"8fd36dbf-f40d-43a5-918e-130ff5128ac4","Type":"ContainerStarted","Data":"f6f67944d3d8df9711109a7b7b70fc05320f9c3a0c360f7c96baadc0919f3e69"}
Jul 06 14:25:16.560199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.560177    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wlxc7" event={"ID":"c3a36fc8-b9bd-47d1-963e-46d65a8867dc","Type":"ContainerStarted","Data":"c892561bd98c99c012f44c07d2332ed1cf89ff3ab1ba13f6b1398a137f528a3d"}
Jul 06 14:25:16.586908 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.586867    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:16.587087 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.587047    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:25:16.587087 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.587070    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-766676f87d-dvlfz: secret "image-registry-tls" not found
Jul 06 14:25:16.587216 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.587126    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls podName:e3bee0b7-abf5-4feb-8da6-dc9d0878114a nodeName:}" failed. No retries permitted until 2026-07-06 14:25:48.587106896 +0000 UTC m=+98.795915272 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls") pod "image-registry-766676f87d-dvlfz" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a") : secret "image-registry-tls" not found
Jul 06 14:25:16.587314 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:16.587265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:16.587435 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.587419    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:25:16.587498 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:16.587475    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls podName:ad56e44f-e089-427b-9532-00d941fa3b80 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:48.587459422 +0000 UTC m=+98.796267809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls") pod "dns-default-vls8r" (UID: "ad56e44f-e089-427b-9532-00d941fa3b80") : secret "dns-default-metrics-tls" not found
Jul 06 14:25:17.001940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.001903    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"]
Jul 06 14:25:17.004025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.004009    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"
Jul 06 14:25:17.007008 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.006950    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 14:25:17.007124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.007075    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 14:25:17.007124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.007075    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q47qw\""
Jul 06 14:25:17.014167 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.014130    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"]
Jul 06 14:25:17.049529 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.049495    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wnznz"]
Jul 06 14:25:17.051789 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.051768    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.060012 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.059969    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 14:25:17.060201 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.060097    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mh8w\""
Jul 06 14:25:17.060405 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.060281    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 14:25:17.069681 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.069652    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wnznz"]
Jul 06 14:25:17.091790 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.091926 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.091926 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c13e903-7cad-423a-b8e7-48a73bc2b241-data-volume\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.092068 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091931    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bltww\" (UniqueName: \"kubernetes.io/projected/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-api-access-bltww\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.092068 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091968    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c13e903-7cad-423a-b8e7-48a73bc2b241-crio-socket\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.092068 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.091985    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lst69\" (UniqueName: \"kubernetes.io/projected/da2dfe16-32aa-421d-b094-4dddab657b2f-kube-api-access-lst69\") pod \"migrator-d57d58cc9-9ktb2\" (UID: \"da2dfe16-32aa-421d-b094-4dddab657b2f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c13e903-7cad-423a-b8e7-48a73bc2b241-data-volume\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193358    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bltww\" (UniqueName: \"kubernetes.io/projected/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-api-access-bltww\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193403    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c13e903-7cad-423a-b8e7-48a73bc2b241-crio-socket\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193432    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lst69\" (UniqueName: \"kubernetes.io/projected/da2dfe16-32aa-421d-b094-4dddab657b2f-kube-api-access-lst69\") pod \"migrator-d57d58cc9-9ktb2\" (UID: \"da2dfe16-32aa-421d-b094-4dddab657b2f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193536    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c13e903-7cad-423a-b8e7-48a73bc2b241-crio-socket\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.193608    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.193654    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.193674    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.693655022 +0000 UTC m=+67.902463387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : configmap "kube-rbac-proxy" not found
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.193683    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c13e903-7cad-423a-b8e7-48a73bc2b241-data-volume\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.193742 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.193702    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:17.693686848 +0000 UTC m=+67.902495213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:17.206286 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.206260    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bltww\" (UniqueName: \"kubernetes.io/projected/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-api-access-bltww\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.206448 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.206357    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lst69\" (UniqueName: \"kubernetes.io/projected/da2dfe16-32aa-421d-b094-4dddab657b2f-kube-api-access-lst69\") pod \"migrator-d57d58cc9-9ktb2\" (UID: \"da2dfe16-32aa-421d-b094-4dddab657b2f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"
Jul 06 14:25:17.315076 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.314999    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"
Jul 06 14:25:17.463249 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.463212    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2"]
Jul 06 14:25:17.466478 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:17.466441    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2dfe16_32aa_421d_b094_4dddab657b2f.slice/crio-94a2f30092245fa919fe1fc8869949530724678cdf3be6981d5a2cff121668f1 WatchSource:0}: Error finding container 94a2f30092245fa919fe1fc8869949530724678cdf3be6981d5a2cff121668f1: Status 404 returned error can't find the container with id 94a2f30092245fa919fe1fc8869949530724678cdf3be6981d5a2cff121668f1
Jul 06 14:25:17.566752 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.566676    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/1.log"
Jul 06 14:25:17.567640 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.567483    2570 scope.go:117] "RemoveContainer" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"
Jul 06 14:25:17.568020 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.567792    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pw8fj_openshift-console-operator(239a5b97-8702-4a74-a4d5-fdad207c183a)\"" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podUID="239a5b97-8702-4a74-a4d5-fdad207c183a"
Jul 06 14:25:17.569063 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.569037    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hf2kl" event={"ID":"8fd36dbf-f40d-43a5-918e-130ff5128ac4","Type":"ContainerStarted","Data":"32950ca54617a81f23b94ae1491c4d426a43b01c30e489e9116bfadd28d30402"}
Jul 06 14:25:17.569555 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.569534    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:25:17.572955 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.572933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2" event={"ID":"da2dfe16-32aa-421d-b094-4dddab657b2f","Type":"ContainerStarted","Data":"94a2f30092245fa919fe1fc8869949530724678cdf3be6981d5a2cff121668f1"}
Jul 06 14:25:17.601328 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.601268    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hf2kl" podStartSLOduration=67.601245656 podStartE2EDuration="1m7.601245656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:17.600744733 +0000 UTC m=+67.809553110" watchObservedRunningTime="2026-07-06 14:25:17.601245656 +0000 UTC m=+67.810054044"
Jul 06 14:25:17.698710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.698671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.698904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:17.698745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:17.698904 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.698818    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 14:25:17.698904 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.698892    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.698871308 +0000 UTC m=+68.907679692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : configmap "kube-rbac-proxy" not found
Jul 06 14:25:17.699395 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.699364    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:17.699528 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:17.699442    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:18.699422157 +0000 UTC m=+68.908230522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:18.102837 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:18.102799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:18.103042 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.102974    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:18.103097 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.103075    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.103053489 +0000 UTC m=+76.311861856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:18.203702 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:18.203664    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:18.203889 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.203835    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:18.203958 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.203932    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.203910406 +0000 UTC m=+76.412718776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:18.708118 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:18.708065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:18.708594 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:18.708134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:18.708594 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.708191    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 14:25:18.708594 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.708238    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:18.708594 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.708255    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:20.708236631 +0000 UTC m=+70.917045016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : configmap "kube-rbac-proxy" not found
Jul 06 14:25:18.708594 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:18.708272    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:20.708261745 +0000 UTC m=+70.917070109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:19.465964 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:19.465932    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b9cph_fc515b02-bb79-4f8d-9122-ca6043180ae8/dns-node-resolver/0.log"
Jul 06 14:25:19.583920 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:19.583882    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2" event={"ID":"da2dfe16-32aa-421d-b094-4dddab657b2f","Type":"ContainerStarted","Data":"15e82a923e782ec06d889b4726c35359997b6019fea156a02c05893780a07277"}
Jul 06 14:25:19.584106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:19.583981    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2" event={"ID":"da2dfe16-32aa-421d-b094-4dddab657b2f","Type":"ContainerStarted","Data":"88411042769d24cdc1f6e0d4809140c34fb276a95b99b96e6c7c7705ca7d5ef4"}
Jul 06 14:25:19.601715 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:19.601656    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9ktb2" podStartSLOduration=2.057994553 podStartE2EDuration="3.601635133s" totalImagesPullingTime="1.54364058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:19.601125692 +0000 UTC m=+69.809934081" watchObservedRunningTime="2026-07-06 14:25:19.601635133 +0000 UTC m=+69.810443522"
Jul 06 14:25:20.574428 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.574383    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:20.574837 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.574479    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:20.574899 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.574885    2570 scope.go:117] "RemoveContainer" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"
Jul 06 14:25:20.575089 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:20.575063    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pw8fj_openshift-console-operator(239a5b97-8702-4a74-a4d5-fdad207c183a)\"" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podUID="239a5b97-8702-4a74-a4d5-fdad207c183a"
Jul 06 14:25:20.667102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.667072    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6vdx_c6ed3cfe-e58e-4b15-ad73-2c821535d27d/node-ca/0.log"
Jul 06 14:25:20.726701 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.726662    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:20.726899 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:20.726755    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 14:25:20.726899 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:20.726825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:20.727030 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:20.726949    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:20.727030 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:20.726971    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:24.726950012 +0000 UTC m=+74.935758387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : configmap "kube-rbac-proxy" not found
Jul 06 14:25:20.727030 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:20.727021    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:24.726985996 +0000 UTC m=+74.935794360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:21.266492 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.266415    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z7n64_29f5a40d-93f2-434a-8da5-624bdbd9b4cc/serve-healthcheck-canary/0.log"
Jul 06 14:25:21.466343 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.466319    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9ktb2_da2dfe16-32aa-421d-b094-4dddab657b2f/migrator/0.log"
Jul 06 14:25:21.591749 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.591712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wlxc7" event={"ID":"c3a36fc8-b9bd-47d1-963e-46d65a8867dc","Type":"ContainerStarted","Data":"a22974d70439280bcb93c8931ebd8571240c69710c34503c6b3558ae35bf3e84"}
Jul 06 14:25:21.608027 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.607866    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wlxc7" podStartSLOduration=67.270973978 podStartE2EDuration="1m11.607848877s" totalImagesPullingTime="4.336874899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:21.607511284 +0000 UTC m=+71.816319674" watchObservedRunningTime="2026-07-06 14:25:21.607848877 +0000 UTC m=+71.816657264"
Jul 06 14:25:21.666445 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.666417    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9ktb2_da2dfe16-32aa-421d-b094-4dddab657b2f/graceful-termination/0.log"
Jul 06 14:25:21.867655 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:21.867579    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-98pgc_9a37022f-52f7-46d1-ba09-448059337885/kube-storage-version-migrator-operator/0.log"
Jul 06 14:25:24.762785 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:24.762748    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:24.762785 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:24.762792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:24.763267 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:24.762877    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 14:25:24.763267 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:24.762942    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:32.762927093 +0000 UTC m=+82.971735461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : configmap "kube-rbac-proxy" not found
Jul 06 14:25:24.763267 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:24.762954    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:24.763267 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:24.763024    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls podName:6c13e903-7cad-423a-b8e7-48a73bc2b241 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:32.762981536 +0000 UTC m=+82.971789900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wnznz" (UID: "6c13e903-7cad-423a-b8e7-48a73bc2b241") : secret "insights-runtime-extractor-tls" not found
Jul 06 14:25:26.173688 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:26.173650    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:26.174107 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:26.173768    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:26.174107 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:26.173825    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls podName:0d65dcd3-852a-4605-86f9-fc9307cffad4 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:42.173812604 +0000 UTC m=+92.382620969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wpkpg" (UID: "0d65dcd3-852a-4605-86f9-fc9307cffad4") : secret "cluster-monitoring-operator-tls" not found
Jul 06 14:25:26.274640 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:26.274597    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:26.274934 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:26.274776    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 14:25:26.274934 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:26.274861    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert podName:142f3bd6-dee6-406f-b7d9-3f7da5786619 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:42.274839634 +0000 UTC m=+92.483648015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qdh88" (UID: "142f3bd6-dee6-406f-b7d9-3f7da5786619") : secret "networking-console-plugin-cert" not found
Jul 06 14:25:32.286953 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.286839    2570 scope.go:117] "RemoveContainer" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"
Jul 06 14:25:32.624880 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.624851    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:25:32.625273 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.625258    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/1.log"
Jul 06 14:25:32.625355 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.625293    2570 generic.go:356] "Generic (PLEG): container finished" podID="239a5b97-8702-4a74-a4d5-fdad207c183a" containerID="048c9992d7101babdf1e7cfd6c2aa9adf700e6642c48c141557d57a8fa8266ae" exitCode=255
Jul 06 14:25:32.625394 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.625355    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" event={"ID":"239a5b97-8702-4a74-a4d5-fdad207c183a","Type":"ContainerDied","Data":"048c9992d7101babdf1e7cfd6c2aa9adf700e6642c48c141557d57a8fa8266ae"}
Jul 06 14:25:32.625429 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.625407    2570 scope.go:117] "RemoveContainer" containerID="e63a33f7c4ac8a1fc13b9a091afcb0106b7d44af27e16822e1c5c73c6ae7200e"
Jul 06 14:25:32.625774 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.625758    2570 scope.go:117] "RemoveContainer" containerID="048c9992d7101babdf1e7cfd6c2aa9adf700e6642c48c141557d57a8fa8266ae"
Jul 06 14:25:32.626007 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:32.625966    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-pw8fj_openshift-console-operator(239a5b97-8702-4a74-a4d5-fdad207c183a)\"" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podUID="239a5b97-8702-4a74-a4d5-fdad207c183a"
Jul 06 14:25:32.829147 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.829110    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:32.829147 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.829155    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:32.829732 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.829709    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c13e903-7cad-423a-b8e7-48a73bc2b241-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:32.831535 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.831513    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c13e903-7cad-423a-b8e7-48a73bc2b241-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wnznz\" (UID: \"6c13e903-7cad-423a-b8e7-48a73bc2b241\") " pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:32.963624 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:32.963537    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wnznz"
Jul 06 14:25:33.087007 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:33.086934    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wnznz"]
Jul 06 14:25:33.089504 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:33.089473    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c13e903_7cad_423a_b8e7_48a73bc2b241.slice/crio-53a5158fe6d3356705861d0e4c978c3a291433284f6588199ba75d9f3623dcd7 WatchSource:0}: Error finding container 53a5158fe6d3356705861d0e4c978c3a291433284f6588199ba75d9f3623dcd7: Status 404 returned error can't find the container with id 53a5158fe6d3356705861d0e4c978c3a291433284f6588199ba75d9f3623dcd7
Jul 06 14:25:33.631038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:33.630933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wnznz" event={"ID":"6c13e903-7cad-423a-b8e7-48a73bc2b241","Type":"ContainerStarted","Data":"53a5158fe6d3356705861d0e4c978c3a291433284f6588199ba75d9f3623dcd7"}
Jul 06 14:25:33.633515 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:33.633433    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:25:37.403155 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.403118    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"]
Jul 06 14:25:37.406369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.406352    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.410106 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.410079    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l2vxr\""
Jul 06 14:25:37.410445 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.410261    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 14:25:37.411556 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.411538    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 14:25:37.411666 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.411649    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 14:25:37.411718 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.411691    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 14:25:37.420411 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.420388    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"]
Jul 06 14:25:37.576759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.576733    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99w9h\" (UniqueName: \"kubernetes.io/projected/7a2b84e1-6623-440c-b98c-0e755a71d5d1-kube-api-access-99w9h\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.576856 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.576793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2b84e1-6623-440c-b98c-0e755a71d5d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.650063 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.650035    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wnznz" event={"ID":"6c13e903-7cad-423a-b8e7-48a73bc2b241","Type":"ContainerStarted","Data":"52f29d63ccb23eb3b79b562e528d4605d43c0273df883e82d02fe8137d33d969"}
Jul 06 14:25:37.650231 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.650087    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wnznz" event={"ID":"6c13e903-7cad-423a-b8e7-48a73bc2b241","Type":"ContainerStarted","Data":"c1ec6b286db5abebae235de25e739049fd10d3ff6a378dbe168c81c837a99126"}
Jul 06 14:25:37.678109 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.678007    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2b84e1-6623-440c-b98c-0e755a71d5d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.678266 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.678169    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99w9h\" (UniqueName: \"kubernetes.io/projected/7a2b84e1-6623-440c-b98c-0e755a71d5d1-kube-api-access-99w9h\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.680547 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.680516    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a2b84e1-6623-440c-b98c-0e755a71d5d1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.692717 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.692681    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99w9h\" (UniqueName: \"kubernetes.io/projected/7a2b84e1-6623-440c-b98c-0e755a71d5d1-kube-api-access-99w9h\") pod \"managed-serviceaccount-addon-agent-76d8b8f944-tfczb\" (UID: \"7a2b84e1-6623-440c-b98c-0e755a71d5d1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.730913 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.730878    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"
Jul 06 14:25:37.861157 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:37.861122    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb"]
Jul 06 14:25:37.865809 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:37.865782    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2b84e1_6623_440c_b98c_0e755a71d5d1.slice/crio-85e64462e2c96b50af4a5d6cb36a1f725b3abd0863c5f3d0e331d2918d9ba2c3 WatchSource:0}: Error finding container 85e64462e2c96b50af4a5d6cb36a1f725b3abd0863c5f3d0e331d2918d9ba2c3: Status 404 returned error can't find the container with id 85e64462e2c96b50af4a5d6cb36a1f725b3abd0863c5f3d0e331d2918d9ba2c3
Jul 06 14:25:38.655323 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:38.655280    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb" event={"ID":"7a2b84e1-6623-440c-b98c-0e755a71d5d1","Type":"ContainerStarted","Data":"85e64462e2c96b50af4a5d6cb36a1f725b3abd0863c5f3d0e331d2918d9ba2c3"}
Jul 06 14:25:40.574768 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:40.574736    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:40.575218 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:40.574802    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:40.575218 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:40.575187    2570 scope.go:117] "RemoveContainer" containerID="048c9992d7101babdf1e7cfd6c2aa9adf700e6642c48c141557d57a8fa8266ae"
Jul 06 14:25:40.575422 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:40.575399    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-pw8fj_openshift-console-operator(239a5b97-8702-4a74-a4d5-fdad207c183a)\"" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podUID="239a5b97-8702-4a74-a4d5-fdad207c183a"
Jul 06 14:25:41.668698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:41.668661    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wnznz" event={"ID":"6c13e903-7cad-423a-b8e7-48a73bc2b241","Type":"ContainerStarted","Data":"f7ba7d9deeadb83d182ff8029ff83acc3cee41fe87ccbab458df667fd88756e3"}
Jul 06 14:25:41.670101 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:41.670072    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb" event={"ID":"7a2b84e1-6623-440c-b98c-0e755a71d5d1","Type":"ContainerStarted","Data":"66d85d5ad4676d6bfb9e36baab2e4be7d154e2fafa23b66af350fa13dc4c5d76"}
Jul 06 14:25:41.687351 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:41.687302    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wnznz" podStartSLOduration=16.63879953 podStartE2EDuration="24.687285017s" totalImagesPullingTime="8.048485487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:25:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:41.686486208 +0000 UTC m=+91.895294597" watchObservedRunningTime="2026-07-06 14:25:41.687285017 +0000 UTC m=+91.896093403"
Jul 06 14:25:41.703697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:41.703651    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76d8b8f944-tfczb" podStartSLOduration=1.258764535 podStartE2EDuration="4.703637137s" totalImagesPullingTime="3.444872602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:41.703071831 +0000 UTC m=+91.911880219" watchObservedRunningTime="2026-07-06 14:25:41.703637137 +0000 UTC m=+91.912445523"
Jul 06 14:25:42.217808 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.217762    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:42.220219 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.220187    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d65dcd3-852a-4605-86f9-fc9307cffad4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wpkpg\" (UID: \"0d65dcd3-852a-4605-86f9-fc9307cffad4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:42.318230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.318190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:42.320662 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.320642    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/142f3bd6-dee6-406f-b7d9-3f7da5786619-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qdh88\" (UID: \"142f3bd6-dee6-406f-b7d9-3f7da5786619\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:42.354879 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.354838    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"
Jul 06 14:25:42.450321 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.450288    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"
Jul 06 14:25:42.484115 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.484022    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg"]
Jul 06 14:25:42.488527 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:42.488495    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d65dcd3_852a_4605_86f9_fc9307cffad4.slice/crio-17db3897d4fed9e875221fe8a76c504308784748ac8c34381738c4a7636e6980 WatchSource:0}: Error finding container 17db3897d4fed9e875221fe8a76c504308784748ac8c34381738c4a7636e6980: Status 404 returned error can't find the container with id 17db3897d4fed9e875221fe8a76c504308784748ac8c34381738c4a7636e6980
Jul 06 14:25:42.583234 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.583199    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qdh88"]
Jul 06 14:25:42.585852 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:42.585825    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142f3bd6_dee6_406f_b7d9_3f7da5786619.slice/crio-2b42b69cfa1ed82b3d0d2956ba3d36c15a78030c038b37e35031fa8728bcc1f0 WatchSource:0}: Error finding container 2b42b69cfa1ed82b3d0d2956ba3d36c15a78030c038b37e35031fa8728bcc1f0: Status 404 returned error can't find the container with id 2b42b69cfa1ed82b3d0d2956ba3d36c15a78030c038b37e35031fa8728bcc1f0
Jul 06 14:25:42.674526 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.674491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88" event={"ID":"142f3bd6-dee6-406f-b7d9-3f7da5786619","Type":"ContainerStarted","Data":"2b42b69cfa1ed82b3d0d2956ba3d36c15a78030c038b37e35031fa8728bcc1f0"}
Jul 06 14:25:42.675724 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:42.675699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg" event={"ID":"0d65dcd3-852a-4605-86f9-fc9307cffad4","Type":"ContainerStarted","Data":"17db3897d4fed9e875221fe8a76c504308784748ac8c34381738c4a7636e6980"}
Jul 06 14:25:44.685770 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:44.685661    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88" event={"ID":"142f3bd6-dee6-406f-b7d9-3f7da5786619","Type":"ContainerStarted","Data":"252870a33a075036d9a80a8559636c8a3bc2ee4b65a47b1603ef35cd61b4efeb"}
Jul 06 14:25:44.687292 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:44.687264    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg" event={"ID":"0d65dcd3-852a-4605-86f9-fc9307cffad4","Type":"ContainerStarted","Data":"3cb50be0797fe58abd2453c12739c55fd882053b4e57528bcf6c29f87dfa03fd"}
Jul 06 14:25:44.703653 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:44.703602    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qdh88" podStartSLOduration=32.923077853 podStartE2EDuration="34.703572172s" totalImagesPullingTime="1.780494319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:44.702846113 +0000 UTC m=+94.911654552" watchObservedRunningTime="2026-07-06 14:25:44.703572172 +0000 UTC m=+94.912380557"
Jul 06 14:25:44.730230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:44.730174    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wpkpg" podStartSLOduration=32.80804189 podStartE2EDuration="34.730151291s" totalImagesPullingTime="1.922109401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:44.729915888 +0000 UTC m=+94.938724294" watchObservedRunningTime="2026-07-06 14:25:44.730151291 +0000 UTC m=+94.938959678"
Jul 06 14:25:48.671069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.671029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:48.671574 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.671090    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:48.673372 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.673348    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad56e44f-e089-427b-9532-00d941fa3b80-metrics-tls\") pod \"dns-default-vls8r\" (UID: \"ad56e44f-e089-427b-9532-00d941fa3b80\") " pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:48.673476 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.673405    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"image-registry-766676f87d-dvlfz\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") " pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:48.887811 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.887773    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9dzg8\""
Jul 06 14:25:48.895408 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.895386    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:48.918655 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.918631    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s54bc\""
Jul 06 14:25:48.926005 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:48.925923    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:49.068430 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.068396    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:25:49.072343 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:49.072318    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3bee0b7_abf5_4feb_8da6_dc9d0878114a.slice/crio-ce954408f4bdf929b32f80efb688a351b7440e88c111740819664d707714d0a3 WatchSource:0}: Error finding container ce954408f4bdf929b32f80efb688a351b7440e88c111740819664d707714d0a3: Status 404 returned error can't find the container with id ce954408f4bdf929b32f80efb688a351b7440e88c111740819664d707714d0a3
Jul 06 14:25:49.086823 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.086794    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vls8r"]
Jul 06 14:25:49.090305 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:49.090279    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad56e44f_e089_427b_9532_00d941fa3b80.slice/crio-8b979f009b54d3f37e489e8f79b6b7c7dde4208bb64ab12e72171100ecbc771d WatchSource:0}: Error finding container 8b979f009b54d3f37e489e8f79b6b7c7dde4208bb64ab12e72171100ecbc771d: Status 404 returned error can't find the container with id 8b979f009b54d3f37e489e8f79b6b7c7dde4208bb64ab12e72171100ecbc771d
Jul 06 14:25:49.587045 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.587010    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hf2kl"
Jul 06 14:25:49.706771 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.706727    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vls8r" event={"ID":"ad56e44f-e089-427b-9532-00d941fa3b80","Type":"ContainerStarted","Data":"8b979f009b54d3f37e489e8f79b6b7c7dde4208bb64ab12e72171100ecbc771d"}
Jul 06 14:25:49.708526 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.708498    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" event={"ID":"e3bee0b7-abf5-4feb-8da6-dc9d0878114a","Type":"ContainerStarted","Data":"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"}
Jul 06 14:25:49.708664 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.708540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" event={"ID":"e3bee0b7-abf5-4feb-8da6-dc9d0878114a","Type":"ContainerStarted","Data":"ce954408f4bdf929b32f80efb688a351b7440e88c111740819664d707714d0a3"}
Jul 06 14:25:49.708664 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.708579    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:25:49.731223 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:49.731146    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" podStartSLOduration=99.731123647 podStartE2EDuration="1m39.731123647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:49.729850321 +0000 UTC m=+99.938658733" watchObservedRunningTime="2026-07-06 14:25:49.731123647 +0000 UTC m=+99.939932035"
Jul 06 14:25:51.718121 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:51.718026    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vls8r" event={"ID":"ad56e44f-e089-427b-9532-00d941fa3b80","Type":"ContainerStarted","Data":"152ac9cca55ad368d45152ddb6b29c7324384d9e067f30c4e290bab5ade6911b"}
Jul 06 14:25:52.723800 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:52.723763    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vls8r" event={"ID":"ad56e44f-e089-427b-9532-00d941fa3b80","Type":"ContainerStarted","Data":"4ed094d003247fd9ddbedac7c0f836785e0ee41241457841e5a7bbba0975d646"}
Jul 06 14:25:52.724219 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:52.723868    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vls8r"
Jul 06 14:25:52.745048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:52.744977    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vls8r" podStartSLOduration=66.369706147 podStartE2EDuration="1m8.744960484s" totalImagesPullingTime="2.375254337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:52.742949375 +0000 UTC m=+102.951757758" watchObservedRunningTime="2026-07-06 14:25:52.744960484 +0000 UTC m=+102.953768848"
Jul 06 14:25:54.287060 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.287027    2570 scope.go:117] "RemoveContainer" containerID="048c9992d7101babdf1e7cfd6c2aa9adf700e6642c48c141557d57a8fa8266ae"
Jul 06 14:25:54.734711 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.734682    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:25:54.734899 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.734787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" event={"ID":"239a5b97-8702-4a74-a4d5-fdad207c183a","Type":"ContainerStarted","Data":"51577f1356492fcda643cf6c8254062426a384dd17847e15a5976ba123feeae4"}
Jul 06 14:25:54.735180 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.735158    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:54.740528 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.740505    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj"
Jul 06 14:25:54.760069 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:54.760017    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-pw8fj" podStartSLOduration=40.308292099 podStartE2EDuration="44.759982525s" totalImagesPullingTime="4.451690426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:54.75894515 +0000 UTC m=+104.967753565" watchObservedRunningTime="2026-07-06 14:25:54.759982525 +0000 UTC m=+104.968790911"
Jul 06 14:25:56.478414 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.478384    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5fpmk"]
Jul 06 14:25:56.481397 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.481375    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.484171 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.484134    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 14:25:56.484306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.484207    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 14:25:56.484482 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.484468    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 14:25:56.484685 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.484673    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 14:25:56.485657 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.485641    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5g2jr\""
Jul 06 14:25:56.531469 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-textfile\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531490    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531535    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmr8c\" (UniqueName: \"kubernetes.io/projected/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-kube-api-access-cmr8c\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531570    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-metrics-client-ca\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531588    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-wtmp\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531606    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-root\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531795 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531795 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531653    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.531795 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.531684    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-sys\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.632950 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.632923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.632961    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmr8c\" (UniqueName: \"kubernetes.io/projected/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-kube-api-access-cmr8c\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.632985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-metrics-client-ca\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-wtmp\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633043    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-root\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:56.633072    2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633090    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633128 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633121    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-sys\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633507 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:25:56.633138    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls podName:d39a271f-1e65-47b0-8b2c-6c558ab5ecd1 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:57.133119876 +0000 UTC m=+107.341928244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls") pod "node-exporter-5fpmk" (UID: "d39a271f-1e65-47b0-8b2c-6c558ab5ecd1") : secret "node-exporter-tls" not found
Jul 06 14:25:56.633507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633161    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-sys\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-textfile\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633215    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-wtmp\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633378    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-root\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633718 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633539    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-textfile\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633809 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633793    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.633912 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.633892    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-metrics-client-ca\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.635748 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.635727    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:56.645359 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:56.645332    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmr8c\" (UniqueName: \"kubernetes.io/projected/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-kube-api-access-cmr8c\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:57.137282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:57.137247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:57.139595 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:57.139566    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39a271f-1e65-47b0-8b2c-6c558ab5ecd1-node-exporter-tls\") pod \"node-exporter-5fpmk\" (UID: \"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1\") " pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:57.391140 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:57.391044    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fpmk"
Jul 06 14:25:57.402561 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:25:57.402533    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39a271f_1e65_47b0_8b2c_6c558ab5ecd1.slice/crio-7e4a21f6c9d970b615236ed01d5ef715ff1e35300db65ddf60f8fb0170006af1 WatchSource:0}: Error finding container 7e4a21f6c9d970b615236ed01d5ef715ff1e35300db65ddf60f8fb0170006af1: Status 404 returned error can't find the container with id 7e4a21f6c9d970b615236ed01d5ef715ff1e35300db65ddf60f8fb0170006af1
Jul 06 14:25:57.747140 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:57.747045    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fpmk" event={"ID":"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1","Type":"ContainerStarted","Data":"7e4a21f6c9d970b615236ed01d5ef715ff1e35300db65ddf60f8fb0170006af1"}
Jul 06 14:25:58.752131 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:58.752044    2570 generic.go:356] "Generic (PLEG): container finished" podID="d39a271f-1e65-47b0-8b2c-6c558ab5ecd1" containerID="07678dbf44dbe9713b76dba2e2053cf75fe78ec071d98ea50548d401c1e0a2bb" exitCode=0
Jul 06 14:25:58.752487 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:58.752128    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fpmk" event={"ID":"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1","Type":"ContainerDied","Data":"07678dbf44dbe9713b76dba2e2053cf75fe78ec071d98ea50548d401c1e0a2bb"}
Jul 06 14:25:59.758348 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:59.758310    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fpmk" event={"ID":"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1","Type":"ContainerStarted","Data":"5f671ca06a251b5be363001d5ea80c54d375d6c41e2c9370f1537b458ce144b7"}
Jul 06 14:25:59.758794 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:59.758364    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fpmk" event={"ID":"d39a271f-1e65-47b0-8b2c-6c558ab5ecd1","Type":"ContainerStarted","Data":"2e7629012335d8a0d313425554069f0e03f87adbeb42af52f375458e0c1b05fd"}
Jul 06 14:25:59.782042 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:25:59.781963    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5fpmk" podStartSLOduration=2.679541575 podStartE2EDuration="3.781943991s" totalImagesPullingTime="1.035643215s" totalInitContainerRuntime="66.759201ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:59.779844054 +0000 UTC m=+109.988652442" watchObservedRunningTime="2026-07-06 14:25:59.781943991 +0000 UTC m=+109.990752377"
Jul 06 14:26:01.341750 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:01.341712    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:26:02.729797 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:02.729768    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vls8r"
Jul 06 14:26:03.717916 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.717880    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:03.720088 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.720070    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.724432 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724411    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 14:26:03.724432 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724420    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kmbkj\""
Jul 06 14:26:03.724602 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724445    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 14:26:03.724602 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724451    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 14:26:03.724744 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724726    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 14:26:03.724815 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724726    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 14:26:03.724815 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.724759    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 14:26:03.725026 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.725004    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 14:26:03.731698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.731660    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:03.795202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795168    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.795414 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.795414 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795321    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.795414 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795353    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.795551 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.795551 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.795471    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d2t\" (UniqueName: \"kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896510 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896530    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896574    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896598    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d2t\" (UniqueName: \"kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.896721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.896646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.897345 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.897320    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.897438 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.897362    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.897438 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.897374    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.899156 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.899126    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.899272 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.899238    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:03.907791 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:03.907763    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d2t\" (UniqueName: \"kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t\") pod \"console-78b9944f9c-5455v\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") " pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:04.029153 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:04.029071    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:04.157392 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:04.157361    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:04.160385 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:26:04.160360    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c27d8e_f094_4eaf_a4e7_4b8fd6df3de3.slice/crio-fdcf6aada552c241f09c9dce64a70abe50d45194022467be08d097eed4dc9a30 WatchSource:0}: Error finding container fdcf6aada552c241f09c9dce64a70abe50d45194022467be08d097eed4dc9a30: Status 404 returned error can't find the container with id fdcf6aada552c241f09c9dce64a70abe50d45194022467be08d097eed4dc9a30
Jul 06 14:26:04.777681 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:04.777645    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b9944f9c-5455v" event={"ID":"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3","Type":"ContainerStarted","Data":"fdcf6aada552c241f09c9dce64a70abe50d45194022467be08d097eed4dc9a30"}
Jul 06 14:26:07.790781 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:07.790743    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b9944f9c-5455v" event={"ID":"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3","Type":"ContainerStarted","Data":"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"}
Jul 06 14:26:07.819152 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:07.819099    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78b9944f9c-5455v" podStartSLOduration=1.8929644570000002 podStartE2EDuration="4.819082219s" totalImagesPullingTime="2.926117762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:07.817632794 +0000 UTC m=+118.026441181" watchObservedRunningTime="2026-07-06 14:26:07.819082219 +0000 UTC m=+118.027890605"
Jul 06 14:26:11.347223 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:11.347194    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:26:13.041272 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.041240    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:26:13.063495 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.063459    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:26:13.063643 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.063561    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.073130 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.073104    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 14:26:13.175893 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.175859    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.175893 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.175895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.176134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.175945    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.176134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.175977    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.176134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.176050    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.176134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.176088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.176269 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.176168    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqx28\" (UniqueName: \"kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277054 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.276987    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqx28\" (UniqueName: \"kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277102    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277170    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277221    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.277891 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277861    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.278037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277906    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.278037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.277952    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.278384 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.278361    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.279891 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.279861    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.279974 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.279864    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.285498 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.285471    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqx28\" (UniqueName: \"kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28\") pod \"console-766597f668-78rsp\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") " pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.373063 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.373027    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:13.510456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.510411    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:26:13.513701 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:26:13.513668    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6658d827_2a12_46ae_9524_d85d145a010a.slice/crio-c8888abe2441a08cfcd1d7e26b99fa06004cdb6f5e05a65abe67cc0f92fdb7b6 WatchSource:0}: Error finding container c8888abe2441a08cfcd1d7e26b99fa06004cdb6f5e05a65abe67cc0f92fdb7b6: Status 404 returned error can't find the container with id c8888abe2441a08cfcd1d7e26b99fa06004cdb6f5e05a65abe67cc0f92fdb7b6
Jul 06 14:26:13.815159 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.815083    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766597f668-78rsp" event={"ID":"6658d827-2a12-46ae-9524-d85d145a010a","Type":"ContainerStarted","Data":"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"}
Jul 06 14:26:13.815159 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.815135    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766597f668-78rsp" event={"ID":"6658d827-2a12-46ae-9524-d85d145a010a","Type":"ContainerStarted","Data":"c8888abe2441a08cfcd1d7e26b99fa06004cdb6f5e05a65abe67cc0f92fdb7b6"}
Jul 06 14:26:13.840796 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:13.840725    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-766597f668-78rsp" podStartSLOduration=0.840708705 podStartE2EDuration="840.708705ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:26:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:13.839486629 +0000 UTC m=+124.048295016" watchObservedRunningTime="2026-07-06 14:26:13.840708705 +0000 UTC m=+124.049517090"
Jul 06 14:26:14.033775 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:14.029261    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:14.033775 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:14.029645    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:14.038287 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:14.038249    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:14.822048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:14.822022    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:20.135782 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.135740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:26:20.138054 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.138020    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b-metrics-certs\") pod \"network-metrics-daemon-smh7l\" (UID: \"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b\") " pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:26:20.218637 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.218600    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-689l7\""
Jul 06 14:26:20.225858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.225836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-smh7l"
Jul 06 14:26:20.364768 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.364732    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-smh7l"]
Jul 06 14:26:20.367941 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:26:20.367915    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bf4c0b_9f61_4d8b_aff5_3c9efe2d6f5b.slice/crio-b89ba23813b01263aa91a6b49bdd201770a753e163d21aba98a082c236fb04df WatchSource:0}: Error finding container b89ba23813b01263aa91a6b49bdd201770a753e163d21aba98a082c236fb04df: Status 404 returned error can't find the container with id b89ba23813b01263aa91a6b49bdd201770a753e163d21aba98a082c236fb04df
Jul 06 14:26:20.838652 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:20.838611    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-smh7l" event={"ID":"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b","Type":"ContainerStarted","Data":"b89ba23813b01263aa91a6b49bdd201770a753e163d21aba98a082c236fb04df"}
Jul 06 14:26:21.843203 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.843171    2570 generic.go:356] "Generic (PLEG): container finished" podID="9a37022f-52f7-46d1-ba09-448059337885" containerID="b65997f4c735245c8e31cfda265234aad283b6e08b36dced4d075dae7cc71810" exitCode=0
Jul 06 14:26:21.843579 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.843246    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc" event={"ID":"9a37022f-52f7-46d1-ba09-448059337885","Type":"ContainerDied","Data":"b65997f4c735245c8e31cfda265234aad283b6e08b36dced4d075dae7cc71810"}
Jul 06 14:26:21.843579 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.843575    2570 scope.go:117] "RemoveContainer" containerID="b65997f4c735245c8e31cfda265234aad283b6e08b36dced4d075dae7cc71810"
Jul 06 14:26:21.845067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.845046    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-smh7l" event={"ID":"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b","Type":"ContainerStarted","Data":"6e3fc4d57975c77f7edeae7b3e7ea5be4b9d69499c80e99275ebdc7c68c3b304"}
Jul 06 14:26:21.845145 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.845086    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-smh7l" event={"ID":"e2bf4c0b-9f61-4d8b-aff5-3c9efe2d6f5b","Type":"ContainerStarted","Data":"8258e54f8fe09783b74d4b92ca6def81b62bb3d155958619023a95c8c1dd099d"}
Jul 06 14:26:21.877858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:21.877474    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-smh7l" podStartSLOduration=130.770471669 podStartE2EDuration="2m11.877453867s" totalImagesPullingTime="1.106982198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:21.876449975 +0000 UTC m=+132.085258364" watchObservedRunningTime="2026-07-06 14:26:21.877453867 +0000 UTC m=+132.086262254"
Jul 06 14:26:22.850718 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:22.850676    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-98pgc" event={"ID":"9a37022f-52f7-46d1-ba09-448059337885","Type":"ContainerStarted","Data":"85ae5d21a52b6b52e58ae323c3a351037a14fa5bc525700867578be67697259a"}
Jul 06 14:26:23.373867 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:23.373833    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:23.374066 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:23.373886    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:23.378734 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:23.378711    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:23.858721 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:23.858690    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:26:23.909071 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:23.909035    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:26.363253 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.363198    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" podUID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" containerName="registry" containerID="cri-o://13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087" gracePeriod=30
Jul 06 14:26:26.611774 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.611749    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:26:26.686820 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686779    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686842    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686865    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686903    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686930    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.686960    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.687006    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh7pb\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687322 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.687076    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") pod \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\" (UID: \"e3bee0b7-abf5-4feb-8da6-dc9d0878114a\") "
Jul 06 14:26:26.687322 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.687294    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:26:26.687411 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.687388    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:26:26.689492 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.689463    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:26:26.689613 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.689559    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:26:26.689693 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.689674    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:26:26.689808 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.689778    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb" (OuterVolumeSpecName: "kube-api-access-dh7pb") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "kube-api-access-dh7pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:26:26.689866 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.689811    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:26:26.696954 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.696927    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e3bee0b7-abf5-4feb-8da6-dc9d0878114a" (UID: "e3bee0b7-abf5-4feb-8da6-dc9d0878114a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787822    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-bound-sa-token\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787856    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-trusted-ca\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787865    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-certificates\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787875    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-installation-pull-secrets\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787884    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-ca-trust-extracted\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787895    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-image-registry-private-configuration\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787905    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh7pb\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-kube-api-access-dh7pb\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.787911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.787916    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3bee0b7-abf5-4feb-8da6-dc9d0878114a-registry-tls\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:26.867090 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.867054    2570 generic.go:356] "Generic (PLEG): container finished" podID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" containerID="13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087" exitCode=0
Jul 06 14:26:26.867267 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.867101    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-766676f87d-dvlfz"
Jul 06 14:26:26.867267 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.867147    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" event={"ID":"e3bee0b7-abf5-4feb-8da6-dc9d0878114a","Type":"ContainerDied","Data":"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"}
Jul 06 14:26:26.867267 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.867193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-766676f87d-dvlfz" event={"ID":"e3bee0b7-abf5-4feb-8da6-dc9d0878114a","Type":"ContainerDied","Data":"ce954408f4bdf929b32f80efb688a351b7440e88c111740819664d707714d0a3"}
Jul 06 14:26:26.867267 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.867219    2570 scope.go:117] "RemoveContainer" containerID="13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"
Jul 06 14:26:26.868897 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.868875    2570 generic.go:356] "Generic (PLEG): container finished" podID="6dda94dd-e4ce-4c50-bb3c-211940025ef0" containerID="5c5fe2c3de0f8aff30ae65f1127338f7a334df3d12567c693a2ea52752230e51" exitCode=0
Jul 06 14:26:26.869017 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.868918    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt" event={"ID":"6dda94dd-e4ce-4c50-bb3c-211940025ef0","Type":"ContainerDied","Data":"5c5fe2c3de0f8aff30ae65f1127338f7a334df3d12567c693a2ea52752230e51"}
Jul 06 14:26:26.869235 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.869223    2570 scope.go:117] "RemoveContainer" containerID="5c5fe2c3de0f8aff30ae65f1127338f7a334df3d12567c693a2ea52752230e51"
Jul 06 14:26:26.880248 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.880222    2570 scope.go:117] "RemoveContainer" containerID="13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"
Jul 06 14:26:26.880575 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:26:26.880554    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087\": container with ID starting with 13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087 not found: ID does not exist" containerID="13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"
Jul 06 14:26:26.880624 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.880584    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087"} err="failed to get container status \"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087\": rpc error: code = NotFound desc = could not find container \"13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087\": container with ID starting with 13d17fa82a83bb3ffe921fc4e9cb75fc8deef9b71a2db5538e624d6c15c02087 not found: ID does not exist"
Jul 06 14:26:26.908115 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.908086    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:26:26.912537 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:26.912502    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-766676f87d-dvlfz"]
Jul 06 14:26:27.875902 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:27.875867    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-sg7jt" event={"ID":"6dda94dd-e4ce-4c50-bb3c-211940025ef0","Type":"ContainerStarted","Data":"be442a80e1266dcdcd5d2a01fc39a1d1b62d863e8bf1dab74579d9eb53eac00f"}
Jul 06 14:26:28.292742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:28.292669    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" path="/var/lib/kubelet/pods/e3bee0b7-abf5-4feb-8da6-dc9d0878114a/volumes"
Jul 06 14:26:48.932315 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:48.932246    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-78b9944f9c-5455v" podUID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" containerName="console" containerID="cri-o://83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102" gracePeriod=15
Jul 06 14:26:49.173979 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.173951    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:49.265943 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.265845    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.265943 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.265891    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.265943 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.265908    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.266247 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.265952    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.266247 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.266023    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.266247 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.266062    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2d2t\" (UniqueName: \"kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t\") pod \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\" (UID: \"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3\") "
Jul 06 14:26:49.266463 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.266422    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:26:49.266526 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.266465    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config" (OuterVolumeSpecName: "console-config") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:26:49.266526 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.266516    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca" (OuterVolumeSpecName: "service-ca") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:26:49.268266 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.268230    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t" (OuterVolumeSpecName: "kube-api-access-j2d2t") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "kube-api-access-j2d2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:26:49.268266 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.268238    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:26:49.268405 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.268266    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" (UID: "95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:26:49.366799 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366758    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-oauth-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.366799 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366792    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.366799 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366805    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2d2t\" (UniqueName: \"kubernetes.io/projected/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-kube-api-access-j2d2t\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.367083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366817    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-service-ca\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.367083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366830    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-console-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.367083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.366840    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3-oauth-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:26:49.959443 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.959409    2570 generic.go:356] "Generic (PLEG): container finished" podID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" containerID="83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102" exitCode=0
Jul 06 14:26:49.959934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.959461    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b9944f9c-5455v"
Jul 06 14:26:49.959934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.959506    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b9944f9c-5455v" event={"ID":"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3","Type":"ContainerDied","Data":"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"}
Jul 06 14:26:49.959934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.959553    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b9944f9c-5455v" event={"ID":"95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3","Type":"ContainerDied","Data":"fdcf6aada552c241f09c9dce64a70abe50d45194022467be08d097eed4dc9a30"}
Jul 06 14:26:49.959934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.959574    2570 scope.go:117] "RemoveContainer" containerID="83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"
Jul 06 14:26:49.968262 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.968244    2570 scope.go:117] "RemoveContainer" containerID="83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"
Jul 06 14:26:49.968527 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:26:49.968511    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102\": container with ID starting with 83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102 not found: ID does not exist" containerID="83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"
Jul 06 14:26:49.968577 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.968537    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102"} err="failed to get container status \"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102\": rpc error: code = NotFound desc = could not find container \"83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102\": container with ID starting with 83f96246c4f11a290e7da8aef8bc3c75cb9bd029141745292e5bc4706c628102 not found: ID does not exist"
Jul 06 14:26:49.982379 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.982344    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:49.986079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:49.986053    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78b9944f9c-5455v"]
Jul 06 14:26:50.291277 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:26:50.291200    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" path="/var/lib/kubelet/pods/95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3/volumes"
Jul 06 14:27:19.507749 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.507711    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.507971    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" containerName="registry"
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.508005    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" containerName="registry"
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.508041    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" containerName="console"
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.508050    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" containerName="console"
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.508123    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3bee0b7-abf5-4feb-8da6-dc9d0878114a" containerName="registry"
Jul 06 14:27:19.508306 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.508136    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c27d8e-f094-4eaf-a4e7-4b8fd6df3de3" containerName="console"
Jul 06 14:27:19.512422 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.512401    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.521046 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.521019    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:27:19.592940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.592906    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.592940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.592948    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.593226 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.592981    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.593226 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.593027    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.593226 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.593086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r44qt\" (UniqueName: \"kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.593226 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.593133    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.593226 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.593193    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694010 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.693958    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694178 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694178 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694178 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694080    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r44qt\" (UniqueName: \"kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694178 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694178 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694494 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.694848 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694815    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.695032 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.695006    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.695152 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694951    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.695152 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.694874    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.696588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.696569    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.696670 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.696591    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.705553 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.705526    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r44qt\" (UniqueName: \"kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt\") pod \"console-684c4dd45d-lrkn8\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") " pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.822304 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.822279    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:19.958521 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:19.958490    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:27:19.961488 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:27:19.961462    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2539e6_2526_4671_9410_4a3fe7db3c26.slice/crio-13bfc7987018ff727604da91f0b9b5f8d5d18fcd4f149f22728d8d35c26b4fba WatchSource:0}: Error finding container 13bfc7987018ff727604da91f0b9b5f8d5d18fcd4f149f22728d8d35c26b4fba: Status 404 returned error can't find the container with id 13bfc7987018ff727604da91f0b9b5f8d5d18fcd4f149f22728d8d35c26b4fba
Jul 06 14:27:20.073002 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:20.072908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684c4dd45d-lrkn8" event={"ID":"1c2539e6-2526-4671-9410-4a3fe7db3c26","Type":"ContainerStarted","Data":"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"}
Jul 06 14:27:20.073002 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:20.072956    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684c4dd45d-lrkn8" event={"ID":"1c2539e6-2526-4671-9410-4a3fe7db3c26","Type":"ContainerStarted","Data":"13bfc7987018ff727604da91f0b9b5f8d5d18fcd4f149f22728d8d35c26b4fba"}
Jul 06 14:27:20.092733 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:20.092658    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-684c4dd45d-lrkn8" podStartSLOduration=1.092640785 podStartE2EDuration="1.092640785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:27:20.092302753 +0000 UTC m=+190.301111387" watchObservedRunningTime="2026-07-06 14:27:20.092640785 +0000 UTC m=+190.301449173"
Jul 06 14:27:29.822435 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:29.822394    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:29.822830 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:29.822459    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:29.827282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:29.827256    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:30.112091 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:30.111982    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:27:30.226566 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:30.226533    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:27:55.249182 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.249118    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-766597f668-78rsp" podUID="6658d827-2a12-46ae-9524-d85d145a010a" containerName="console" containerID="cri-o://42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a" gracePeriod=15
Jul 06 14:27:55.493900 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.493872    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:27:55.679838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679791    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.679838 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679844    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679886    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679905    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679939    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqx28\" (UniqueName: \"kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.679967    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.680024    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle\") pod \"6658d827-2a12-46ae-9524-d85d145a010a\" (UID: \"6658d827-2a12-46ae-9524-d85d145a010a\") "
Jul 06 14:27:55.680453 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.680418    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:27:55.680569 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.680518    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:27:55.680634 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.680554    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:27:55.680634 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.680597    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config" (OuterVolumeSpecName: "console-config") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:27:55.682196 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.682167    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:27:55.682309 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.682285    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:27:55.682309 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.682301    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28" (OuterVolumeSpecName: "kube-api-access-wqx28") pod "6658d827-2a12-46ae-9524-d85d145a010a" (UID: "6658d827-2a12-46ae-9524-d85d145a010a"). InnerVolumeSpecName "kube-api-access-wqx28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:27:55.780894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780840    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-oauth-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.780894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780887    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-oauth-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.780894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780897    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqx28\" (UniqueName: \"kubernetes.io/projected/6658d827-2a12-46ae-9524-d85d145a010a-kube-api-access-wqx28\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.780894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780908    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-console-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.780894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780917    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-trusted-ca-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.781231 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780925    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6658d827-2a12-46ae-9524-d85d145a010a-service-ca\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:55.781231 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:55.780933    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6658d827-2a12-46ae-9524-d85d145a010a-console-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:27:56.200877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.200842    2570 generic.go:356] "Generic (PLEG): container finished" podID="6658d827-2a12-46ae-9524-d85d145a010a" containerID="42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a" exitCode=0
Jul 06 14:27:56.201067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.200888    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766597f668-78rsp"
Jul 06 14:27:56.201067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.200889    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766597f668-78rsp" event={"ID":"6658d827-2a12-46ae-9524-d85d145a010a","Type":"ContainerDied","Data":"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"}
Jul 06 14:27:56.201067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.201029    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766597f668-78rsp" event={"ID":"6658d827-2a12-46ae-9524-d85d145a010a","Type":"ContainerDied","Data":"c8888abe2441a08cfcd1d7e26b99fa06004cdb6f5e05a65abe67cc0f92fdb7b6"}
Jul 06 14:27:56.201067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.201060    2570 scope.go:117] "RemoveContainer" containerID="42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"
Jul 06 14:27:56.209754 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.209738    2570 scope.go:117] "RemoveContainer" containerID="42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"
Jul 06 14:27:56.210034 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:27:56.210017    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a\": container with ID starting with 42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a not found: ID does not exist" containerID="42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"
Jul 06 14:27:56.210079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.210044    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a"} err="failed to get container status \"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a\": rpc error: code = NotFound desc = could not find container \"42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a\": container with ID starting with 42ef793af3a988cad37f136899db0116c4b3dd294808d8e63967d85a5afdfb1a not found: ID does not exist"
Jul 06 14:27:56.222756 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.222725    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:27:56.226643 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.226620    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-766597f668-78rsp"]
Jul 06 14:27:56.291086 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:27:56.291057    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6658d827-2a12-46ae-9524-d85d145a010a" path="/var/lib/kubelet/pods/6658d827-2a12-46ae-9524-d85d145a010a/volumes"
Jul 06 14:28:06.496023 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.495967    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"]
Jul 06 14:28:06.496391 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.496249    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6658d827-2a12-46ae-9524-d85d145a010a" containerName="console"
Jul 06 14:28:06.496391 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.496261    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6658d827-2a12-46ae-9524-d85d145a010a" containerName="console"
Jul 06 14:28:06.496391 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.496307    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6658d827-2a12-46ae-9524-d85d145a010a" containerName="console"
Jul 06 14:28:06.499202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.499187    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.502134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.502113    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 14:28:06.503463 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.503424    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l5rvq\""
Jul 06 14:28:06.503566 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.503465    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 14:28:06.509384 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.509361    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"]
Jul 06 14:28:06.555198 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.555163    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bbnx\" (UniqueName: \"kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.555365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.555224    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.555365 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.555287    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.655592 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.655553    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.655802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.655608    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.655802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.655638    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bbnx\" (UniqueName: \"kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.655931 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.655908    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.656203 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.655963    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.665033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.665004    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bbnx\" (UniqueName: \"kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.808393 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.808297    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:06.938142 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:06.938112    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"]
Jul 06 14:28:06.941093 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:06.941066    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895c33a7_022c_4d58_8e03_c8ce6000c52d.slice/crio-e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207 WatchSource:0}: Error finding container e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207: Status 404 returned error can't find the container with id e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207
Jul 06 14:28:07.244590 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:07.244557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg" event={"ID":"895c33a7-022c-4d58-8e03-c8ce6000c52d","Type":"ContainerStarted","Data":"e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207"}
Jul 06 14:28:15.274315 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:15.274275    2570 generic.go:356] "Generic (PLEG): container finished" podID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerID="b51164c2071e5ae9a2e611af2e18d33ac51020263b842ad421412869c996c225" exitCode=0
Jul 06 14:28:15.274716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:15.274332    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg" event={"ID":"895c33a7-022c-4d58-8e03-c8ce6000c52d","Type":"ContainerDied","Data":"b51164c2071e5ae9a2e611af2e18d33ac51020263b842ad421412869c996c225"}
Jul 06 14:28:18.287115 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:18.287081    2570 generic.go:356] "Generic (PLEG): container finished" podID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerID="26381627d812fffb83b89691aed8be2e2ab32470003a089eb8dac42e4b800591" exitCode=0
Jul 06 14:28:18.290885 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:18.290858    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg" event={"ID":"895c33a7-022c-4d58-8e03-c8ce6000c52d","Type":"ContainerDied","Data":"26381627d812fffb83b89691aed8be2e2ab32470003a089eb8dac42e4b800591"}
Jul 06 14:28:27.324232 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:27.324194    2570 generic.go:356] "Generic (PLEG): container finished" podID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerID="231485a0ef3462b0a23f555bfe569e0716b904e49d622a8b61e76abf4f87a5b9" exitCode=0
Jul 06 14:28:27.324652 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:27.324282    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg" event={"ID":"895c33a7-022c-4d58-8e03-c8ce6000c52d","Type":"ContainerDied","Data":"231485a0ef3462b0a23f555bfe569e0716b904e49d622a8b61e76abf4f87a5b9"}
Jul 06 14:28:28.506176 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.506141    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:28.634696 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.634610    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util\") pod \"895c33a7-022c-4d58-8e03-c8ce6000c52d\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") "
Jul 06 14:28:28.634696 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.634657    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle\") pod \"895c33a7-022c-4d58-8e03-c8ce6000c52d\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") "
Jul 06 14:28:28.635105 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.634712    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bbnx\" (UniqueName: \"kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx\") pod \"895c33a7-022c-4d58-8e03-c8ce6000c52d\" (UID: \"895c33a7-022c-4d58-8e03-c8ce6000c52d\") "
Jul 06 14:28:28.635323 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.635295    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle" (OuterVolumeSpecName: "bundle") pod "895c33a7-022c-4d58-8e03-c8ce6000c52d" (UID: "895c33a7-022c-4d58-8e03-c8ce6000c52d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:28:28.636870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.636848    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx" (OuterVolumeSpecName: "kube-api-access-2bbnx") pod "895c33a7-022c-4d58-8e03-c8ce6000c52d" (UID: "895c33a7-022c-4d58-8e03-c8ce6000c52d"). InnerVolumeSpecName "kube-api-access-2bbnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:28:28.639202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.639182    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util" (OuterVolumeSpecName: "util") pod "895c33a7-022c-4d58-8e03-c8ce6000c52d" (UID: "895c33a7-022c-4d58-8e03-c8ce6000c52d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:28:28.735967 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.735935    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:28.735967 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.735963    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895c33a7-022c-4d58-8e03-c8ce6000c52d-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:28.735967 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:28.735974    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bbnx\" (UniqueName: \"kubernetes.io/projected/895c33a7-022c-4d58-8e03-c8ce6000c52d-kube-api-access-2bbnx\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:29.333215 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:29.333176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg" event={"ID":"895c33a7-022c-4d58-8e03-c8ce6000c52d","Type":"ContainerDied","Data":"e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207"}
Jul 06 14:28:29.333390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:29.333223    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4dg"
Jul 06 14:28:29.333390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:29.333225    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2008f120eb9debd04101bce481407ff8bc88dc4ba907ed27474e30d52971207"
Jul 06 14:28:34.579624 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579538    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"]
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579835    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="extract"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579849    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="extract"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579871    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="pull"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579876    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="pull"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579884    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="util"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579889    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="util"
Jul 06 14:28:34.580126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.579929    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="895c33a7-022c-4d58-8e03-c8ce6000c52d" containerName="extract"
Jul 06 14:28:34.581867 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.581851    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.585238 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.585205    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:28:34.585366 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.585285    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 06 14:28:34.585458 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.585442    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ntf7j\""
Jul 06 14:28:34.597424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.597401    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"]
Jul 06 14:28:34.678918 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.678881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803fbe6b-98be-4b81-99f7-288b2ef63905-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.679119 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.679023    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-849t6\" (UniqueName: \"kubernetes.io/projected/803fbe6b-98be-4b81-99f7-288b2ef63905-kube-api-access-849t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.779854 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.779795    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803fbe6b-98be-4b81-99f7-288b2ef63905-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.780102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.779918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-849t6\" (UniqueName: \"kubernetes.io/projected/803fbe6b-98be-4b81-99f7-288b2ef63905-kube-api-access-849t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.780251 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.780226    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803fbe6b-98be-4b81-99f7-288b2ef63905-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.793591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.793559    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-849t6\" (UniqueName: \"kubernetes.io/projected/803fbe6b-98be-4b81-99f7-288b2ef63905-kube-api-access-849t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-h25h9\" (UID: \"803fbe6b-98be-4b81-99f7-288b2ef63905\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:34.890875 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:34.890778    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"
Jul 06 14:28:35.024237 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:35.024200    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9"]
Jul 06 14:28:35.028657 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:35.028629    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803fbe6b_98be_4b81_99f7_288b2ef63905.slice/crio-7bf7e5b585cdab106180692fe6188f4aa4a2ca0353e7cb5add1c13164dc0e705 WatchSource:0}: Error finding container 7bf7e5b585cdab106180692fe6188f4aa4a2ca0353e7cb5add1c13164dc0e705: Status 404 returned error can't find the container with id 7bf7e5b585cdab106180692fe6188f4aa4a2ca0353e7cb5add1c13164dc0e705
Jul 06 14:28:35.356228 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:35.356193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9" event={"ID":"803fbe6b-98be-4b81-99f7-288b2ef63905","Type":"ContainerStarted","Data":"7bf7e5b585cdab106180692fe6188f4aa4a2ca0353e7cb5add1c13164dc0e705"}
Jul 06 14:28:37.364858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:37.364822    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9" event={"ID":"803fbe6b-98be-4b81-99f7-288b2ef63905","Type":"ContainerStarted","Data":"37b6df19a183488186ac63329c0a8b27f47314daf4f2bb163c2abff03e7efb79"}
Jul 06 14:28:37.420209 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:37.420150    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h25h9" podStartSLOduration=2.063987708 podStartE2EDuration="3.420132473s" totalImagesPullingTime="1.356144765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:37.417387704 +0000 UTC m=+267.626196108" watchObservedRunningTime="2026-07-06 14:28:37.420132473 +0000 UTC m=+267.628940858"
Jul 06 14:28:38.485963 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.485925    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"]
Jul 06 14:28:38.488465 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.488447    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.491355 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.491334    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l5rvq\""
Jul 06 14:28:38.491566 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.491552    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 14:28:38.492621 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.492606    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 14:28:38.498598 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.498577    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"]
Jul 06 14:28:38.609558 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.609515    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.609754 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.609571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hc7\" (UniqueName: \"kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.609754 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.609628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.710338 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.710284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.710548 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.710353    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hc7\" (UniqueName: \"kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.710548 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.710402    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.710687 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.710670    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.710781 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.710762    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.720333 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.720300    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hc7\" (UniqueName: \"kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.797707 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.797607    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:38.834820 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.834781    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-flk7x"]
Jul 06 14:28:38.838086 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.838060    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:38.841290 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.841261    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-z6xbs\""
Jul 06 14:28:38.841290 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.841276    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 14:28:38.841482 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.841384    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 14:28:38.851716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.851684    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-flk7x"]
Jul 06 14:28:38.911456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.911422    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74kqg\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-kube-api-access-74kqg\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:38.911616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.911500    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:38.934504 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:38.934474    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"]
Jul 06 14:28:38.937602 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:38.937571    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ead7563_054f_4661_98bc_6add6ab6c225.slice/crio-30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630 WatchSource:0}: Error finding container 30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630: Status 404 returned error can't find the container with id 30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630
Jul 06 14:28:39.012509 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.012480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:39.012646 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.012533    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74kqg\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-kube-api-access-74kqg\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:39.021879 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.021848    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74kqg\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-kube-api-access-74kqg\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:39.023633 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.023608    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fbd9f23-780c-47ba-9507-c3593fb4150f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-flk7x\" (UID: \"9fbd9f23-780c-47ba-9507-c3593fb4150f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:39.148140 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.148097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:39.281076 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.281030    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-flk7x"]
Jul 06 14:28:39.283552 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:39.283527    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fbd9f23_780c_47ba_9507_c3593fb4150f.slice/crio-80342021b0f434d3cbb101d63c7e7b25f99d052b2cc00940246e92787d36f325 WatchSource:0}: Error finding container 80342021b0f434d3cbb101d63c7e7b25f99d052b2cc00940246e92787d36f325: Status 404 returned error can't find the container with id 80342021b0f434d3cbb101d63c7e7b25f99d052b2cc00940246e92787d36f325
Jul 06 14:28:39.374389 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.374356    2570 generic.go:356] "Generic (PLEG): container finished" podID="2ead7563-054f-4661-98bc-6add6ab6c225" containerID="b1f2e02873af8411f6586396a5ad6ae9ad058ac3e88cb4528711172ea4d1830c" exitCode=0
Jul 06 14:28:39.374570 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.374437    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796" event={"ID":"2ead7563-054f-4661-98bc-6add6ab6c225","Type":"ContainerDied","Data":"b1f2e02873af8411f6586396a5ad6ae9ad058ac3e88cb4528711172ea4d1830c"}
Jul 06 14:28:39.374570 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.374483    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796" event={"ID":"2ead7563-054f-4661-98bc-6add6ab6c225","Type":"ContainerStarted","Data":"30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630"}
Jul 06 14:28:39.375656 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:39.375636    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x" event={"ID":"9fbd9f23-780c-47ba-9507-c3593fb4150f","Type":"ContainerStarted","Data":"80342021b0f434d3cbb101d63c7e7b25f99d052b2cc00940246e92787d36f325"}
Jul 06 14:28:43.395258 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:43.395222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x" event={"ID":"9fbd9f23-780c-47ba-9507-c3593fb4150f","Type":"ContainerStarted","Data":"71b000313621dfcb6cbd37e67c1a422887de46e6278f16e928803f3a8cfb5540"}
Jul 06 14:28:43.395731 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:43.395376    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:43.396944 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:43.396921    2570 generic.go:356] "Generic (PLEG): container finished" podID="2ead7563-054f-4661-98bc-6add6ab6c225" containerID="2cd0925e68bc313c47211ee5d03df7e401ef365a3094dc24bcb757b9db8eefb5" exitCode=0
Jul 06 14:28:43.397071 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:43.396957    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796" event={"ID":"2ead7563-054f-4661-98bc-6add6ab6c225","Type":"ContainerDied","Data":"2cd0925e68bc313c47211ee5d03df7e401ef365a3094dc24bcb757b9db8eefb5"}
Jul 06 14:28:43.418598 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:43.418548    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x" podStartSLOduration=2.233496511 podStartE2EDuration="5.418530419s" totalImagesPullingTime="3.185033908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:43.417796665 +0000 UTC m=+273.626605048" watchObservedRunningTime="2026-07-06 14:28:43.418530419 +0000 UTC m=+273.627338805"
Jul 06 14:28:44.402862 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:44.402825    2570 generic.go:356] "Generic (PLEG): container finished" podID="2ead7563-054f-4661-98bc-6add6ab6c225" containerID="9fb91ec755d0dd22cfb471ee8ee06f96e6bc37a3f6eb3f28fbbc6f52fec77119" exitCode=0
Jul 06 14:28:44.403328 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:44.402906    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796" event={"ID":"2ead7563-054f-4661-98bc-6add6ab6c225","Type":"ContainerDied","Data":"9fb91ec755d0dd22cfb471ee8ee06f96e6bc37a3f6eb3f28fbbc6f52fec77119"}
Jul 06 14:28:45.398167 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.398133    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"]
Jul 06 14:28:45.402985 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.402962    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.405639 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.405614    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jljnn\""
Jul 06 14:28:45.409473 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.409429    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"]
Jul 06 14:28:45.466554 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.466519    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.466871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.466755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlps\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-kube-api-access-cqlps\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.532840 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.532817    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:45.567710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.567674    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlps\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-kube-api-access-cqlps\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.567876 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.567773    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.576884 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.576859    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.577043 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.576951    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlps\" (UniqueName: \"kubernetes.io/projected/804c4ef6-07e8-48af-a551-7e85142b4528-kube-api-access-cqlps\") pod \"cert-manager-cainjector-559cdd8c87-gbp7n\" (UID: \"804c4ef6-07e8-48af-a551-7e85142b4528\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.668390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.668304    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util\") pod \"2ead7563-054f-4661-98bc-6add6ab6c225\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") "
Jul 06 14:28:45.668390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.668384    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4hc7\" (UniqueName: \"kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7\") pod \"2ead7563-054f-4661-98bc-6add6ab6c225\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") "
Jul 06 14:28:45.668565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.668425    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle\") pod \"2ead7563-054f-4661-98bc-6add6ab6c225\" (UID: \"2ead7563-054f-4661-98bc-6add6ab6c225\") "
Jul 06 14:28:45.668885 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.668853    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle" (OuterVolumeSpecName: "bundle") pod "2ead7563-054f-4661-98bc-6add6ab6c225" (UID: "2ead7563-054f-4661-98bc-6add6ab6c225"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:28:45.670546 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.670506    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7" (OuterVolumeSpecName: "kube-api-access-g4hc7") pod "2ead7563-054f-4661-98bc-6add6ab6c225" (UID: "2ead7563-054f-4661-98bc-6add6ab6c225"). InnerVolumeSpecName "kube-api-access-g4hc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:28:45.672853 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.672833    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util" (OuterVolumeSpecName: "util") pod "2ead7563-054f-4661-98bc-6add6ab6c225" (UID: "2ead7563-054f-4661-98bc-6add6ab6c225"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:28:45.713676 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.713643    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"
Jul 06 14:28:45.769114 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.769081    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:45.769114 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.769114    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ead7563-054f-4661-98bc-6add6ab6c225-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:45.769335 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.769127    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4hc7\" (UniqueName: \"kubernetes.io/projected/2ead7563-054f-4661-98bc-6add6ab6c225-kube-api-access-g4hc7\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:28:45.841046 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:45.841006    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n"]
Jul 06 14:28:45.843786 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:45.843754    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod804c4ef6_07e8_48af_a551_7e85142b4528.slice/crio-2a14c549a355f8a81cc1aa1945c63736317e651fdafb3a0f2f81c5957f718154 WatchSource:0}: Error finding container 2a14c549a355f8a81cc1aa1945c63736317e651fdafb3a0f2f81c5957f718154: Status 404 returned error can't find the container with id 2a14c549a355f8a81cc1aa1945c63736317e651fdafb3a0f2f81c5957f718154
Jul 06 14:28:46.412330 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.412296    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n" event={"ID":"804c4ef6-07e8-48af-a551-7e85142b4528","Type":"ContainerStarted","Data":"88443780546a59c8ca302b555d6f4f1f63412e392afdc8a89850e5d808b16203"}
Jul 06 14:28:46.412330 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.412338    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n" event={"ID":"804c4ef6-07e8-48af-a551-7e85142b4528","Type":"ContainerStarted","Data":"2a14c549a355f8a81cc1aa1945c63736317e651fdafb3a0f2f81c5957f718154"}
Jul 06 14:28:46.414233 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.414211    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796" event={"ID":"2ead7563-054f-4661-98bc-6add6ab6c225","Type":"ContainerDied","Data":"30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630"}
Jul 06 14:28:46.414350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.414246    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30237b9632526e56f619331ebcea01ff7ef71dabf8d6c20a603be1a2b5627630"
Jul 06 14:28:46.414350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.414255    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftk796"
Jul 06 14:28:46.428059 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:46.428004    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gbp7n" podStartSLOduration=1.427971921 podStartE2EDuration="1.427971921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:46.42769145 +0000 UTC m=+276.636499829" watchObservedRunningTime="2026-07-06 14:28:46.427971921 +0000 UTC m=+276.636780312"
Jul 06 14:28:49.405697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:49.405665    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-flk7x"
Jul 06 14:28:51.214584 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.214542    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvmzl"]
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.214919    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="extract"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.214937    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="extract"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.214970    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="pull"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.214978    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="pull"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.215009    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="util"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.215018    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="util"
Jul 06 14:28:51.215102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.215092    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ead7563-054f-4661-98bc-6add6ab6c225" containerName="extract"
Jul 06 14:28:51.219295 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.219271    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.223265 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.223244    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7hgnt\""
Jul 06 14:28:51.228964 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.228941    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvmzl"]
Jul 06 14:28:51.320330 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.320290    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rpfq\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-kube-api-access-9rpfq\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.320330 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.320335    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.421438 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.421399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rpfq\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-kube-api-access-9rpfq\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.421438 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.421444    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.433186 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.433152    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rpfq\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-kube-api-access-9rpfq\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.433301 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.433249    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bc91ff0-4cfd-48fb-8e94-f6db6603df53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvmzl\" (UID: \"8bc91ff0-4cfd-48fb-8e94-f6db6603df53\") " pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.528574 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.528488    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wvmzl"
Jul 06 14:28:51.699074 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:51.699043    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvmzl"]
Jul 06 14:28:51.701811 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:51.701771    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bc91ff0_4cfd_48fb_8e94_f6db6603df53.slice/crio-d97b135e4c8fbb3e4fc19c69e93767e6fa37e0b69a4bc03ee33bcc8d4178121d WatchSource:0}: Error finding container d97b135e4c8fbb3e4fc19c69e93767e6fa37e0b69a4bc03ee33bcc8d4178121d: Status 404 returned error can't find the container with id d97b135e4c8fbb3e4fc19c69e93767e6fa37e0b69a4bc03ee33bcc8d4178121d
Jul 06 14:28:52.441120 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:52.441081    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wvmzl" event={"ID":"8bc91ff0-4cfd-48fb-8e94-f6db6603df53","Type":"ContainerStarted","Data":"6d5ab53f1d5431e07031c1f06eff9d14852fe02b7419854eb5028b4b88102d79"}
Jul 06 14:28:52.441458 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:52.441128    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wvmzl" event={"ID":"8bc91ff0-4cfd-48fb-8e94-f6db6603df53","Type":"ContainerStarted","Data":"d97b135e4c8fbb3e4fc19c69e93767e6fa37e0b69a4bc03ee33bcc8d4178121d"}
Jul 06 14:28:59.326283 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.326226    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wvmzl" podStartSLOduration=8.326206109 podStartE2EDuration="8.326206109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:52.473706793 +0000 UTC m=+282.682515179" watchObservedRunningTime="2026-07-06 14:28:59.326206109 +0000 UTC m=+289.535014495"
Jul 06 14:28:59.327025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.326981    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"]
Jul 06 14:28:59.331172 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.331151    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.335113 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.335085    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l5rvq\""
Jul 06 14:28:59.335234 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.335094    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 14:28:59.336272 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.336253    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 14:28:59.344067 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.344046    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"]
Jul 06 14:28:59.490761 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.490730    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.490761 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.490767    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.490959 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.490827    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wkw\" (UniqueName: \"kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.592037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.591930    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96wkw\" (UniqueName: \"kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.592037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.591975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.592254 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.592036    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.592442 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.592424    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.592511 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.592453    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.602478 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.602448    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wkw\" (UniqueName: \"kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.640595 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.640559    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:28:59.773907 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:28:59.773876    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"]
Jul 06 14:28:59.776895 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:28:59.776865    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0907b07_0737_4ee8_b7d6_89754d8d87fa.slice/crio-b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a WatchSource:0}: Error finding container b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a: Status 404 returned error can't find the container with id b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a
Jul 06 14:29:00.476007 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:00.475965    2570 generic.go:356] "Generic (PLEG): container finished" podID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerID="466de8c2bad227ba1e82b85e70b84af6c444d9b7238d4abe549bd1fc30f15f9d" exitCode=0
Jul 06 14:29:00.476398 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:00.476057    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq" event={"ID":"b0907b07-0737-4ee8-b7d6-89754d8d87fa","Type":"ContainerDied","Data":"466de8c2bad227ba1e82b85e70b84af6c444d9b7238d4abe549bd1fc30f15f9d"}
Jul 06 14:29:00.476398 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:00.476106    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq" event={"ID":"b0907b07-0737-4ee8-b7d6-89754d8d87fa","Type":"ContainerStarted","Data":"b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a"}
Jul 06 14:29:01.481579 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:01.481542    2570 generic.go:356] "Generic (PLEG): container finished" podID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerID="201029ca42cefbfae6fc964a848cf188b317c6789efad6496f7809185fb05f2d" exitCode=0
Jul 06 14:29:01.481958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:01.481626    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq" event={"ID":"b0907b07-0737-4ee8-b7d6-89754d8d87fa","Type":"ContainerDied","Data":"201029ca42cefbfae6fc964a848cf188b317c6789efad6496f7809185fb05f2d"}
Jul 06 14:29:02.487795 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:02.487761    2570 generic.go:356] "Generic (PLEG): container finished" podID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerID="b01110fa458e920e65d8a4361e34cc4a52f9bc2a8bf4a2d5a9ac97af43cb172e" exitCode=0
Jul 06 14:29:02.488190 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:02.487853    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq" event={"ID":"b0907b07-0737-4ee8-b7d6-89754d8d87fa","Type":"ContainerDied","Data":"b01110fa458e920e65d8a4361e34cc4a52f9bc2a8bf4a2d5a9ac97af43cb172e"}
Jul 06 14:29:03.617197 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.617175    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:29:03.727523 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.727490    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle\") pod \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") "
Jul 06 14:29:03.727697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.727555    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96wkw\" (UniqueName: \"kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw\") pod \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") "
Jul 06 14:29:03.727697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.727595    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util\") pod \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\" (UID: \"b0907b07-0737-4ee8-b7d6-89754d8d87fa\") "
Jul 06 14:29:03.728537 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.728495    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle" (OuterVolumeSpecName: "bundle") pod "b0907b07-0737-4ee8-b7d6-89754d8d87fa" (UID: "b0907b07-0737-4ee8-b7d6-89754d8d87fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:29:03.729722 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.729689    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw" (OuterVolumeSpecName: "kube-api-access-96wkw") pod "b0907b07-0737-4ee8-b7d6-89754d8d87fa" (UID: "b0907b07-0737-4ee8-b7d6-89754d8d87fa"). InnerVolumeSpecName "kube-api-access-96wkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:29:03.733600 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.733573    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util" (OuterVolumeSpecName: "util") pod "b0907b07-0737-4ee8-b7d6-89754d8d87fa" (UID: "b0907b07-0737-4ee8-b7d6-89754d8d87fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:29:03.828797 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.828769    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:29:03.828940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.828799    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96wkw\" (UniqueName: \"kubernetes.io/projected/b0907b07-0737-4ee8-b7d6-89754d8d87fa-kube-api-access-96wkw\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:29:03.828940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:03.828810    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0907b07-0737-4ee8-b7d6-89754d8d87fa-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:29:04.498640 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:04.498613    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq"
Jul 06 14:29:04.498801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:04.498640    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc96nq" event={"ID":"b0907b07-0737-4ee8-b7d6-89754d8d87fa","Type":"ContainerDied","Data":"b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a"}
Jul 06 14:29:04.498801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:04.498680    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0011e0c0badccacb1d0a208eebc6a2191b8b67244fc5d69bc730ac476638a8a"
Jul 06 14:29:10.171092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:10.171051    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:29:10.171667 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:10.171644    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:29:10.181048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:10.181026    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 14:29:16.931967 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.931923    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"]
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932325    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="pull"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932343    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="pull"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932376    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="extract"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932385    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="extract"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932393    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="util"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932402    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="util"
Jul 06 14:29:16.933440 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.932459    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0907b07-0737-4ee8-b7d6-89754d8d87fa" containerName="extract"
Jul 06 14:29:16.934268 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.934250    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:16.939683 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.939625    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 14:29:16.939683 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.939646    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 14:29:16.939870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.939702    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 14:29:16.939870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.939793    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 14:29:16.940072 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.940058    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qlzj2\""
Jul 06 14:29:16.967304 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:16.967267    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"]
Jul 06 14:29:17.027637 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.027598    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.027810 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.027654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.027810 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.027710    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.027810 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.027770    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.027915 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.027809    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb5wf\" (UniqueName: \"kubernetes.io/projected/ab11157e-a292-4369-bf5c-be0013bfaef2-kube-api-access-lb5wf\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.128949 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.128911    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.129154 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.128955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.129154 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.129016    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb5wf\" (UniqueName: \"kubernetes.io/projected/ab11157e-a292-4369-bf5c-be0013bfaef2-kube-api-access-lb5wf\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.129154 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.129045    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.129154 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.129101    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.129374 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.129329    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-manifests\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.131443 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.131410    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab11157e-a292-4369-bf5c-be0013bfaef2-tmp\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.131565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.131519    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.131565 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.131530    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab11157e-a292-4369-bf5c-be0013bfaef2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.149155 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.149132    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb5wf\" (UniqueName: \"kubernetes.io/projected/ab11157e-a292-4369-bf5c-be0013bfaef2-kube-api-access-lb5wf\") pod \"opendatahub-operator-controller-manager-9447c645c-flqn8\" (UID: \"ab11157e-a292-4369-bf5c-be0013bfaef2\") " pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.244419 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.244325    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:17.384135 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.384101    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"]
Jul 06 14:29:17.387579 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:29:17.387551    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab11157e_a292_4369_bf5c_be0013bfaef2.slice/crio-d2cba3a30eb70bde68f9489323237f0f51581178733fa72e3b680b1f0a17d4a2 WatchSource:0}: Error finding container d2cba3a30eb70bde68f9489323237f0f51581178733fa72e3b680b1f0a17d4a2: Status 404 returned error can't find the container with id d2cba3a30eb70bde68f9489323237f0f51581178733fa72e3b680b1f0a17d4a2
Jul 06 14:29:17.389397 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.389378    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:29:17.552619 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:17.552530    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8" event={"ID":"ab11157e-a292-4369-bf5c-be0013bfaef2","Type":"ContainerStarted","Data":"d2cba3a30eb70bde68f9489323237f0f51581178733fa72e3b680b1f0a17d4a2"}
Jul 06 14:29:20.569696 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:20.569665    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8" event={"ID":"ab11157e-a292-4369-bf5c-be0013bfaef2","Type":"ContainerStarted","Data":"9af2c09f3485155fd15825a02337f47a69b9928f76a407e55080f7c8ff0d7bb5"}
Jul 06 14:29:21.575911 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:21.575868    2570 generic.go:356] "Generic (PLEG): container finished" podID="ab11157e-a292-4369-bf5c-be0013bfaef2" containerID="9af2c09f3485155fd15825a02337f47a69b9928f76a407e55080f7c8ff0d7bb5" exitCode=0
Jul 06 14:29:21.576432 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:21.575929    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8" event={"ID":"ab11157e-a292-4369-bf5c-be0013bfaef2","Type":"ContainerDied","Data":"9af2c09f3485155fd15825a02337f47a69b9928f76a407e55080f7c8ff0d7bb5"}
Jul 06 14:29:22.582922 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:22.582881    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8" event={"ID":"ab11157e-a292-4369-bf5c-be0013bfaef2","Type":"ContainerStarted","Data":"c13044d1f573155a91360e2208fd8df541957cadea5686c738dc7f53e61b676c"}
Jul 06 14:29:22.583425 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:22.583044    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:22.619528 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:22.619474    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8" podStartSLOduration=2.2502095349999998 podStartE2EDuration="6.619457667s" totalImagesPullingTime="3.28746283s" totalInitContainerRuntime="1.081785302s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:22.617340079 +0000 UTC m=+312.826148500" watchObservedRunningTime="2026-07-06 14:29:22.619457667 +0000 UTC m=+312.828266053"
Jul 06 14:29:33.590523 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:33.590487    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9447c645c-flqn8"
Jul 06 14:29:36.557629 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.557587    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kb6f4"]
Jul 06 14:29:36.564657 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.564631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:36.567844 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.567818    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 14:29:36.568794 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.568774    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-prmg2\""
Jul 06 14:29:36.578056 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.578023    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kb6f4"]
Jul 06 14:29:36.693519 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.693487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:36.693672 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.693544    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62p9\" (UniqueName: \"kubernetes.io/projected/ae0395e6-8234-4c18-9476-b9929fadc979-kube-api-access-w62p9\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:36.794161 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.794130    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:36.794338 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.794183    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w62p9\" (UniqueName: \"kubernetes.io/projected/ae0395e6-8234-4c18-9476-b9929fadc979-kube-api-access-w62p9\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:36.794338 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:29:36.794280    2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 14:29:36.794419 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:29:36.794346    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert podName:ae0395e6-8234-4c18-9476-b9929fadc979 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:37.294327676 +0000 UTC m=+327.503136054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert") pod "odh-model-controller-7998c67dc4-kb6f4" (UID: "ae0395e6-8234-4c18-9476-b9929fadc979") : secret "odh-model-controller-webhook-cert" not found
Jul 06 14:29:36.803482 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:36.803456    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62p9\" (UniqueName: \"kubernetes.io/projected/ae0395e6-8234-4c18-9476-b9929fadc979-kube-api-access-w62p9\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:37.298731 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.298691    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:37.301635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.301606    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae0395e6-8234-4c18-9476-b9929fadc979-cert\") pod \"odh-model-controller-7998c67dc4-kb6f4\" (UID: \"ae0395e6-8234-4c18-9476-b9929fadc979\") " pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:37.339075 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.339041    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mkl2l"]
Jul 06 14:29:37.341524 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.341502    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.344239 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.344218    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 14:29:37.354778 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.354754    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mkl2l"]
Jul 06 14:29:37.400013 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.399941    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkmbh\" (UniqueName: \"kubernetes.io/projected/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-kube-api-access-qkmbh\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.400204 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.400106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.477321 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.477254    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:37.501484 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.501454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkmbh\" (UniqueName: \"kubernetes.io/projected/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-kube-api-access-qkmbh\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.501655 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.501544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.501724 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:29:37.501663    2570 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 14:29:37.501784 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:29:37.501733    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs podName:95cc674f-1537-4e2a-b4b6-ddf361f4b5ff nodeName:}" failed. No retries permitted until 2026-07-06 14:29:38.001712624 +0000 UTC m=+328.210521002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs") pod "model-serving-api-54d89bd5fc-mkl2l" (UID: "95cc674f-1537-4e2a-b4b6-ddf361f4b5ff") : secret "model-serving-api-tls" not found
Jul 06 14:29:37.511668 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.511640    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkmbh\" (UniqueName: \"kubernetes.io/projected/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-kube-api-access-qkmbh\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:37.613868 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.613845    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kb6f4"]
Jul 06 14:29:37.615618 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:29:37.615588    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0395e6_8234_4c18_9476_b9929fadc979.slice/crio-97f1154a53debcdd8c9ba43b3c8de300d7e08a9c6bd9df6a41193107efb250e8 WatchSource:0}: Error finding container 97f1154a53debcdd8c9ba43b3c8de300d7e08a9c6bd9df6a41193107efb250e8: Status 404 returned error can't find the container with id 97f1154a53debcdd8c9ba43b3c8de300d7e08a9c6bd9df6a41193107efb250e8
Jul 06 14:29:37.650488 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:37.650455    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4" event={"ID":"ae0395e6-8234-4c18-9476-b9929fadc979","Type":"ContainerStarted","Data":"97f1154a53debcdd8c9ba43b3c8de300d7e08a9c6bd9df6a41193107efb250e8"}
Jul 06 14:29:38.006681 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:38.006645    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:38.009117 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:38.009093    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95cc674f-1537-4e2a-b4b6-ddf361f4b5ff-tls-certs\") pod \"model-serving-api-54d89bd5fc-mkl2l\" (UID: \"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff\") " pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:38.252326 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:38.252294    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:38.440558 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:38.440406    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mkl2l"]
Jul 06 14:29:38.443262 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:29:38.443229    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95cc674f_1537_4e2a_b4b6_ddf361f4b5ff.slice/crio-7a70c32170fb4465d17b6e2444fbfa83ecf44a63b55ab2dc31c808d9044d470c WatchSource:0}: Error finding container 7a70c32170fb4465d17b6e2444fbfa83ecf44a63b55ab2dc31c808d9044d470c: Status 404 returned error can't find the container with id 7a70c32170fb4465d17b6e2444fbfa83ecf44a63b55ab2dc31c808d9044d470c
Jul 06 14:29:38.659596 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:38.659519    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l" event={"ID":"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff","Type":"ContainerStarted","Data":"7a70c32170fb4465d17b6e2444fbfa83ecf44a63b55ab2dc31c808d9044d470c"}
Jul 06 14:29:41.620083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.620054    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"]
Jul 06 14:29:41.622567 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.622547    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.626877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.626858    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w5lh8\""
Jul 06 14:29:41.626877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.626868    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 14:29:41.627079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.627013    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 14:29:41.627079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.627022    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 14:29:41.627079 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.627030    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 14:29:41.634123 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.634103    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"]
Jul 06 14:29:41.678798 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.678758    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l" event={"ID":"95cc674f-1537-4e2a-b4b6-ddf361f4b5ff","Type":"ContainerStarted","Data":"21ce79d30e039953a8a4d92921c0dacf344785f72c4c34e3286fb4e320b7e2c1"}
Jul 06 14:29:41.679017 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.678829    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:41.680484 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.680454    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4" event={"ID":"ae0395e6-8234-4c18-9476-b9929fadc979","Type":"ContainerStarted","Data":"4f20c06f0fe38846c8a194a0e19e02bedd6d7057eceebb24998c7f6fff17de15"}
Jul 06 14:29:41.680626 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.680583    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:41.698464 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.698417    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l" podStartSLOduration=2.379160421 podStartE2EDuration="4.698402815s" totalImagesPullingTime="2.319242394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:41.696702093 +0000 UTC m=+331.905510506" watchObservedRunningTime="2026-07-06 14:29:41.698402815 +0000 UTC m=+331.907211179"
Jul 06 14:29:41.714587 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.714468    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4" podStartSLOduration=2.5652138730000003 podStartE2EDuration="5.714448366s" totalImagesPullingTime="3.149234493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:41.714337105 +0000 UTC m=+331.923145519" watchObservedRunningTime="2026-07-06 14:29:41.714448366 +0000 UTC m=+331.923256754"
Jul 06 14:29:41.739124 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.739092    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tmp\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.739281 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.739226    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqnng\" (UniqueName: \"kubernetes.io/projected/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-kube-api-access-zqnng\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.739388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.739321    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tls-certs\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.840056 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.839979    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tls-certs\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.840239 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.840073    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tmp\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.840239 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.840193    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqnng\" (UniqueName: \"kubernetes.io/projected/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-kube-api-access-zqnng\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.842367 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.842332    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tmp\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.842527 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.842512    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-tls-certs\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.851553 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.851526    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqnng\" (UniqueName: \"kubernetes.io/projected/151c4ae4-f7fe-4bdd-8cd6-3919ded59618-kube-api-access-zqnng\") pod \"kube-auth-proxy-5d848d496-gfwqq\" (UID: \"151c4ae4-f7fe-4bdd-8cd6-3919ded59618\") " pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:41.932793 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:41.932704    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"
Jul 06 14:29:42.084123 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:42.084077    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d848d496-gfwqq"]
Jul 06 14:29:42.086727 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:29:42.086694    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151c4ae4_f7fe_4bdd_8cd6_3919ded59618.slice/crio-d2480d82ce98d336e90431ecb626ffb7f4a13ce01bfc559f272553e633b14869 WatchSource:0}: Error finding container d2480d82ce98d336e90431ecb626ffb7f4a13ce01bfc559f272553e633b14869: Status 404 returned error can't find the container with id d2480d82ce98d336e90431ecb626ffb7f4a13ce01bfc559f272553e633b14869
Jul 06 14:29:42.687703 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:42.687662    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq" event={"ID":"151c4ae4-f7fe-4bdd-8cd6-3919ded59618","Type":"ContainerStarted","Data":"d2480d82ce98d336e90431ecb626ffb7f4a13ce01bfc559f272553e633b14869"}
Jul 06 14:29:44.445964 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.445930    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"]
Jul 06 14:29:44.448910 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.448889    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.454098 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.454075    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 14:29:44.455213 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.455187    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-htqht\""
Jul 06 14:29:44.455473 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.455450    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 14:29:44.455578 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.455539    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 14:29:44.455982 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.455952    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:29:44.458083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.458068    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 14:29:44.477591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.477562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"]
Jul 06 14:29:44.566797 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.566764    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11f5157f-f23c-4067-a47f-e947fc5dc36b-manager-config\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.567038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.566825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-metrics-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.567038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.566866    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.567038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.567022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgtvc\" (UniqueName: \"kubernetes.io/projected/11f5157f-f23c-4067-a47f-e947fc5dc36b-kube-api-access-xgtvc\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.668381 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.668343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11f5157f-f23c-4067-a47f-e947fc5dc36b-manager-config\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.668588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.668405    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-metrics-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.668588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.668438    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.668588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.668561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtvc\" (UniqueName: \"kubernetes.io/projected/11f5157f-f23c-4067-a47f-e947fc5dc36b-kube-api-access-xgtvc\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.669232 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.669180    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/11f5157f-f23c-4067-a47f-e947fc5dc36b-manager-config\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.671516 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.671468    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.671636 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.671556    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/11f5157f-f23c-4067-a47f-e947fc5dc36b-metrics-cert\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.678447 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.678420    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtvc\" (UniqueName: \"kubernetes.io/projected/11f5157f-f23c-4067-a47f-e947fc5dc36b-kube-api-access-xgtvc\") pod \"lws-controller-manager-5b6868f88d-n58v5\" (UID: \"11f5157f-f23c-4067-a47f-e947fc5dc36b\") " pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.760498 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.760469    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:44.910916 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:44.910887    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"]
Jul 06 14:29:44.913940 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:29:44.913880    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f5157f_f23c_4067_a47f_e947fc5dc36b.slice/crio-8192b7ef311aa04cda3ee6e6f20319b2f8ff4a11403ab59feb129a9fadca32e0 WatchSource:0}: Error finding container 8192b7ef311aa04cda3ee6e6f20319b2f8ff4a11403ab59feb129a9fadca32e0: Status 404 returned error can't find the container with id 8192b7ef311aa04cda3ee6e6f20319b2f8ff4a11403ab59feb129a9fadca32e0
Jul 06 14:29:45.704114 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:45.704037    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq" event={"ID":"151c4ae4-f7fe-4bdd-8cd6-3919ded59618","Type":"ContainerStarted","Data":"43e317352459bafd869e783a20daa4d7885078a4f1fa06c229829ded416f74ed"}
Jul 06 14:29:45.705482 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:45.705449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5" event={"ID":"11f5157f-f23c-4067-a47f-e947fc5dc36b","Type":"ContainerStarted","Data":"8192b7ef311aa04cda3ee6e6f20319b2f8ff4a11403ab59feb129a9fadca32e0"}
Jul 06 14:29:45.722942 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:45.722876    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d848d496-gfwqq" podStartSLOduration=2.079629779 podStartE2EDuration="4.722855048s" totalImagesPullingTime="2.643225269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:45.722192675 +0000 UTC m=+335.931001063" watchObservedRunningTime="2026-07-06 14:29:45.722855048 +0000 UTC m=+335.931663435"
Jul 06 14:29:46.711614 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:46.711568    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5" event={"ID":"11f5157f-f23c-4067-a47f-e947fc5dc36b","Type":"ContainerStarted","Data":"8cd9d30484c86f8cb32d3a531f9e730aab769b52bcdb9afae7505985afbae010"}
Jul 06 14:29:46.712032 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:46.711641    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:29:46.741751 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:46.741700    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5" podStartSLOduration=1.42061842 podStartE2EDuration="2.741686035s" totalImagesPullingTime="1.321067615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:46.74048286 +0000 UTC m=+336.949291272" watchObservedRunningTime="2026-07-06 14:29:46.741686035 +0000 UTC m=+336.950494421"
Jul 06 14:29:52.690338 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:52.690298    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kb6f4"
Jul 06 14:29:52.694455 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:52.694432    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mkl2l"
Jul 06 14:29:57.718716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:29:57.718680    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b6868f88d-n58v5"
Jul 06 14:30:52.954648 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.954607    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"]
Jul 06 14:30:52.957044 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.957023    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:52.959931 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.959903    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 14:30:52.960139 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.960044    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 14:30:52.960139 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.960088    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c89dr\""
Jul 06 14:30:52.970136 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:52.970108    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"]
Jul 06 14:30:53.028717 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.028678    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.028904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.028729    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.028904 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.028834    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzdc\" (UniqueName: \"kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.130209 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.130161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.130209 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.130215    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.130474 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.130265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzdc\" (UniqueName: \"kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.130589 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.130568    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.130667 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.130626    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.140499 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.140466    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzdc\" (UniqueName: \"kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.267109 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.267022    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:53.425197 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.425168    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"]
Jul 06 14:30:53.427290 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:30:53.427264    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb488e0f_4ed2_4b35_9fae_40d3e88f3d78.slice/crio-6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990 WatchSource:0}: Error finding container 6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990: Status 404 returned error can't find the container with id 6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990
Jul 06 14:30:53.560383 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.560350    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"]
Jul 06 14:30:53.562759 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.562741    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.577910 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.577877    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"]
Jul 06 14:30:53.634963 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.634916    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.635150 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.634979    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwtqc\" (UniqueName: \"kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.635150 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.635044    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.736268 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.736229    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwtqc\" (UniqueName: \"kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.736442 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.736281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.736442 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.736331    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.736687 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.736672    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.736722 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.736704    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.746580 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.746555    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwtqc\" (UniqueName: \"kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.894959 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.894920    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:53.957557 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.954035    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"]
Jul 06 14:30:53.962803 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.962776    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:53.966961 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:53.966928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"]
Jul 06 14:30:54.018914 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.018881    2570 generic.go:356] "Generic (PLEG): container finished" podID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerID="7f50f2b74ec29fcf32db42e8f8c27b7ff973cf5aa4356f48814465696789ba6a" exitCode=0
Jul 06 14:30:54.019107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.018976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w" event={"ID":"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78","Type":"ContainerDied","Data":"7f50f2b74ec29fcf32db42e8f8c27b7ff973cf5aa4356f48814465696789ba6a"}
Jul 06 14:30:54.019107 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.019048    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w" event={"ID":"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78","Type":"ContainerStarted","Data":"6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990"}
Jul 06 14:30:54.038965 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.038927    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qnr\" (UniqueName: \"kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.039162 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.038982    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.039162 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.039052    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.048778 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.048749    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"]
Jul 06 14:30:54.051540 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:30:54.051510    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d53eb7a_257f_4cfd_af88_6165d8616467.slice/crio-7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e WatchSource:0}: Error finding container 7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e: Status 404 returned error can't find the container with id 7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e
Jul 06 14:30:54.140206 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.140170    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.140390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.140235    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.140390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.140324    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qnr\" (UniqueName: \"kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.140622 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.140595    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.140622 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.140613    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.153147 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.153120    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qnr\" (UniqueName: \"kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.273881 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.273842    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:54.415112 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.415088    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"]
Jul 06 14:30:54.416716 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:30:54.416688    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99c0895_6050_4811_bf21_509bec6bdfdc.slice/crio-f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946 WatchSource:0}: Error finding container f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946: Status 404 returned error can't find the container with id f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946
Jul 06 14:30:54.546763 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.546732    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"]
Jul 06 14:30:54.549401 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.549381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.564518 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.564485    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"]
Jul 06 14:30:54.644678 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.644651    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.644773 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.644755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.644855 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.644793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg8cl\" (UniqueName: \"kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.746160 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.746071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.746160 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.746125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.746160 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.746146    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg8cl\" (UniqueName: \"kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.746468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.746448    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.746543 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.746520    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.756634 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.756599    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg8cl\" (UniqueName: \"kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:54.900962 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:54.900922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:55.025489 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.025410    2570 generic.go:356] "Generic (PLEG): container finished" podID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerID="3bd5dcab041e92eb73534455e72d9bd8dfae338439aedf8861ca934119a70eab" exitCode=0
Jul 06 14:30:55.025951 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.025492    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7" event={"ID":"e99c0895-6050-4811-bf21-509bec6bdfdc","Type":"ContainerDied","Data":"3bd5dcab041e92eb73534455e72d9bd8dfae338439aedf8861ca934119a70eab"}
Jul 06 14:30:55.025951 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.025548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7" event={"ID":"e99c0895-6050-4811-bf21-509bec6bdfdc","Type":"ContainerStarted","Data":"f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946"}
Jul 06 14:30:55.027435 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.027409    2570 generic.go:356] "Generic (PLEG): container finished" podID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerID="3ab7a1278abbc6ade714fd8d1a12854b5066bcf8ffb86e6b560fa9b3d7eb6d4a" exitCode=0
Jul 06 14:30:55.027557 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.027491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w" event={"ID":"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78","Type":"ContainerDied","Data":"3ab7a1278abbc6ade714fd8d1a12854b5066bcf8ffb86e6b560fa9b3d7eb6d4a"}
Jul 06 14:30:55.029031 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.029013    2570 generic.go:356] "Generic (PLEG): container finished" podID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerID="0a4cd7a68fd55cf0002457f5070fbb4771c0076bf9fa34fb063440c5709110a0" exitCode=0
Jul 06 14:30:55.029102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.029065    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz" event={"ID":"7d53eb7a-257f-4cfd-af88-6165d8616467","Type":"ContainerDied","Data":"0a4cd7a68fd55cf0002457f5070fbb4771c0076bf9fa34fb063440c5709110a0"}
Jul 06 14:30:55.029102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.029094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz" event={"ID":"7d53eb7a-257f-4cfd-af88-6165d8616467","Type":"ContainerStarted","Data":"7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e"}
Jul 06 14:30:55.044472 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:55.044448    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"]
Jul 06 14:30:55.047792 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:30:55.047761    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec98fb49_f08c_4a2d_b985_54f95f6110ad.slice/crio-4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497 WatchSource:0}: Error finding container 4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497: Status 404 returned error can't find the container with id 4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497
Jul 06 14:30:56.036307 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.036223    2570 generic.go:356] "Generic (PLEG): container finished" podID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerID="78c1c46f3891498ad511a5601e4929d5aacdcdc47b2bdfad3217eb1bbaa30e9f" exitCode=0
Jul 06 14:30:56.036691 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.036309    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz" event={"ID":"7d53eb7a-257f-4cfd-af88-6165d8616467","Type":"ContainerDied","Data":"78c1c46f3891498ad511a5601e4929d5aacdcdc47b2bdfad3217eb1bbaa30e9f"}
Jul 06 14:30:56.037978 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.037958    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerID="c0bf3f667630d9c5c07e7d51992272cf1897a8afb169dc25c5e49f5ca372da9a" exitCode=0
Jul 06 14:30:56.038078 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.038004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56" event={"ID":"ec98fb49-f08c-4a2d-b985-54f95f6110ad","Type":"ContainerDied","Data":"c0bf3f667630d9c5c07e7d51992272cf1897a8afb169dc25c5e49f5ca372da9a"}
Jul 06 14:30:56.038078 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.038054    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56" event={"ID":"ec98fb49-f08c-4a2d-b985-54f95f6110ad","Type":"ContainerStarted","Data":"4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497"}
Jul 06 14:30:56.039903 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.039880    2570 generic.go:356] "Generic (PLEG): container finished" podID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerID="5b96859d27cdecdc8d566ed0ed23612d336263c1ee2cfdb0f940012c542fd5f2" exitCode=0
Jul 06 14:30:56.040034 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.039924    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7" event={"ID":"e99c0895-6050-4811-bf21-509bec6bdfdc","Type":"ContainerDied","Data":"5b96859d27cdecdc8d566ed0ed23612d336263c1ee2cfdb0f940012c542fd5f2"}
Jul 06 14:30:56.042353 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.042334    2570 generic.go:356] "Generic (PLEG): container finished" podID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerID="8e92ba4c885dba6490164d8f69968477d77b01d284b4dddb75e9fb87ceeff7d8" exitCode=0
Jul 06 14:30:56.042444 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:56.042379    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w" event={"ID":"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78","Type":"ContainerDied","Data":"8e92ba4c885dba6490164d8f69968477d77b01d284b4dddb75e9fb87ceeff7d8"}
Jul 06 14:30:57.049683 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.049650    2570 generic.go:356] "Generic (PLEG): container finished" podID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerID="f6f378f39763fd945e191b1cf9b96734e1a103e94e24f0c200c8c008dc8c8b40" exitCode=0
Jul 06 14:30:57.050146 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.049731    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz" event={"ID":"7d53eb7a-257f-4cfd-af88-6165d8616467","Type":"ContainerDied","Data":"f6f378f39763fd945e191b1cf9b96734e1a103e94e24f0c200c8c008dc8c8b40"}
Jul 06 14:30:57.051413 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.051390    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerID="84b0499962128a7853e9416cf1c75b2191a624df6a050b38e4ee0cbeb7e2a231" exitCode=0
Jul 06 14:30:57.051535 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.051460    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56" event={"ID":"ec98fb49-f08c-4a2d-b985-54f95f6110ad","Type":"ContainerDied","Data":"84b0499962128a7853e9416cf1c75b2191a624df6a050b38e4ee0cbeb7e2a231"}
Jul 06 14:30:57.053583 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.053562    2570 generic.go:356] "Generic (PLEG): container finished" podID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerID="a0c87e39177f03abc93b73c6359d77e3005f1736bd3be02f41edc3f8adf2cbf5" exitCode=0
Jul 06 14:30:57.053697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.053628    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7" event={"ID":"e99c0895-6050-4811-bf21-509bec6bdfdc","Type":"ContainerDied","Data":"a0c87e39177f03abc93b73c6359d77e3005f1736bd3be02f41edc3f8adf2cbf5"}
Jul 06 14:30:57.244984 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.244957    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:57.368392 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.368296    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util\") pod \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") "
Jul 06 14:30:57.368392 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.368337    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzzdc\" (UniqueName: \"kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc\") pod \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") "
Jul 06 14:30:57.368622 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.368403    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle\") pod \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\" (UID: \"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78\") "
Jul 06 14:30:57.368933 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.368911    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle" (OuterVolumeSpecName: "bundle") pod "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" (UID: "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:57.370727 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.370705    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc" (OuterVolumeSpecName: "kube-api-access-fzzdc") pod "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" (UID: "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78"). InnerVolumeSpecName "kube-api-access-fzzdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:30:57.373494 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.373452    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util" (OuterVolumeSpecName: "util") pod "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" (UID: "bb488e0f-4ed2-4b35-9fae-40d3e88f3d78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:57.469650 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.469603    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:57.469650 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.469637    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:57.469855 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:57.469652    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzzdc\" (UniqueName: \"kubernetes.io/projected/bb488e0f-4ed2-4b35-9fae-40d3e88f3d78-kube-api-access-fzzdc\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.060202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.060171    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w"
Jul 06 14:30:58.060202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.060180    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w" event={"ID":"bb488e0f-4ed2-4b35-9fae-40d3e88f3d78","Type":"ContainerDied","Data":"6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990"}
Jul 06 14:30:58.060698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.060264    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6536a0da3e0145c65b4ee2f3b3654cdfce7817ed20196c84cdb37285edd09990"
Jul 06 14:30:58.062725 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.062693    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerID="fb9b579a8088341a44187ea8580c51f6dbbc5dbc36ea30b8e3bd42c5ad742229" exitCode=0
Jul 06 14:30:58.062877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.062755    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56" event={"ID":"ec98fb49-f08c-4a2d-b985-54f95f6110ad","Type":"ContainerDied","Data":"fb9b579a8088341a44187ea8580c51f6dbbc5dbc36ea30b8e3bd42c5ad742229"}
Jul 06 14:30:58.203614 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.203589    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:58.238468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.238440    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:58.377399 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377363    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle\") pod \"e99c0895-6050-4811-bf21-509bec6bdfdc\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") "
Jul 06 14:30:58.377604 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377409    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util\") pod \"7d53eb7a-257f-4cfd-af88-6165d8616467\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") "
Jul 06 14:30:58.377604 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377428    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util\") pod \"e99c0895-6050-4811-bf21-509bec6bdfdc\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") "
Jul 06 14:30:58.377604 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377468    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56qnr\" (UniqueName: \"kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr\") pod \"e99c0895-6050-4811-bf21-509bec6bdfdc\" (UID: \"e99c0895-6050-4811-bf21-509bec6bdfdc\") "
Jul 06 14:30:58.377604 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377500    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle\") pod \"7d53eb7a-257f-4cfd-af88-6165d8616467\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") "
Jul 06 14:30:58.377604 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.377558    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwtqc\" (UniqueName: \"kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc\") pod \"7d53eb7a-257f-4cfd-af88-6165d8616467\" (UID: \"7d53eb7a-257f-4cfd-af88-6165d8616467\") "
Jul 06 14:30:58.378113 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.378078    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle" (OuterVolumeSpecName: "bundle") pod "e99c0895-6050-4811-bf21-509bec6bdfdc" (UID: "e99c0895-6050-4811-bf21-509bec6bdfdc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:58.378296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.378268    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle" (OuterVolumeSpecName: "bundle") pod "7d53eb7a-257f-4cfd-af88-6165d8616467" (UID: "7d53eb7a-257f-4cfd-af88-6165d8616467"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:58.379784 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.379759    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr" (OuterVolumeSpecName: "kube-api-access-56qnr") pod "e99c0895-6050-4811-bf21-509bec6bdfdc" (UID: "e99c0895-6050-4811-bf21-509bec6bdfdc"). InnerVolumeSpecName "kube-api-access-56qnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:30:58.379888 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.379839    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc" (OuterVolumeSpecName: "kube-api-access-mwtqc") pod "7d53eb7a-257f-4cfd-af88-6165d8616467" (UID: "7d53eb7a-257f-4cfd-af88-6165d8616467"). InnerVolumeSpecName "kube-api-access-mwtqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:30:58.382736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.382696    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util" (OuterVolumeSpecName: "util") pod "e99c0895-6050-4811-bf21-509bec6bdfdc" (UID: "e99c0895-6050-4811-bf21-509bec6bdfdc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:58.383342 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.383322    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util" (OuterVolumeSpecName: "util") pod "7d53eb7a-257f-4cfd-af88-6165d8616467" (UID: "7d53eb7a-257f-4cfd-af88-6165d8616467"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:58.478901 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478861    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwtqc\" (UniqueName: \"kubernetes.io/projected/7d53eb7a-257f-4cfd-af88-6165d8616467-kube-api-access-mwtqc\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.478901 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478896    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.478901 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478905    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.478901 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478913    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99c0895-6050-4811-bf21-509bec6bdfdc-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.479199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478921    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56qnr\" (UniqueName: \"kubernetes.io/projected/e99c0895-6050-4811-bf21-509bec6bdfdc-kube-api-access-56qnr\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:58.479199 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:58.478931    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d53eb7a-257f-4cfd-af88-6165d8616467-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:59.069177 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.069150    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7"
Jul 06 14:30:59.069566 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.069183    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7" event={"ID":"e99c0895-6050-4811-bf21-509bec6bdfdc","Type":"ContainerDied","Data":"f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946"}
Jul 06 14:30:59.069566 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.069228    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4716527e43f08ce6b7c5fd3d86f9131d7b10adbf1e78aba6987219d575a3946"
Jul 06 14:30:59.071131 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.071103    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz"
Jul 06 14:30:59.071230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.071125    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz" event={"ID":"7d53eb7a-257f-4cfd-af88-6165d8616467","Type":"ContainerDied","Data":"7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e"}
Jul 06 14:30:59.071230 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.071168    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d8c89c4aa018f51dafe6835191175a56e3cca463e020e3b45ac71b6a636404e"
Jul 06 14:30:59.202885 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.202859    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:30:59.284455 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.284418    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util\") pod \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") "
Jul 06 14:30:59.284455 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.284462    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg8cl\" (UniqueName: \"kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl\") pod \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") "
Jul 06 14:30:59.284689 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.284505    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle\") pod \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\" (UID: \"ec98fb49-f08c-4a2d-b985-54f95f6110ad\") "
Jul 06 14:30:59.284968 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.284945    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle" (OuterVolumeSpecName: "bundle") pod "ec98fb49-f08c-4a2d-b985-54f95f6110ad" (UID: "ec98fb49-f08c-4a2d-b985-54f95f6110ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:59.286610 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.286578    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl" (OuterVolumeSpecName: "kube-api-access-mg8cl") pod "ec98fb49-f08c-4a2d-b985-54f95f6110ad" (UID: "ec98fb49-f08c-4a2d-b985-54f95f6110ad"). InnerVolumeSpecName "kube-api-access-mg8cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:30:59.290037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.289984    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util" (OuterVolumeSpecName: "util") pod "ec98fb49-f08c-4a2d-b985-54f95f6110ad" (UID: "ec98fb49-f08c-4a2d-b985-54f95f6110ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:30:59.385204 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.385109    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-util\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:59.385204 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.385143    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg8cl\" (UniqueName: \"kubernetes.io/projected/ec98fb49-f08c-4a2d-b985-54f95f6110ad-kube-api-access-mg8cl\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:30:59.385204 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:30:59.385153    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec98fb49-f08c-4a2d-b985-54f95f6110ad-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:31:00.078193 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:31:00.078164    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56"
Jul 06 14:31:00.078677 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:31:00.078164    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56" event={"ID":"ec98fb49-f08c-4a2d-b985-54f95f6110ad","Type":"ContainerDied","Data":"4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497"}
Jul 06 14:31:00.078677 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:31:00.078279    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f500b6da1170c855aa6a4f93397f34983aca40685149dbea3f5059f2b8f1497"
Jul 06 14:32:29.048350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048314    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048618    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048628    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048637    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048642    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048649    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048654    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048659    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048664    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048672    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048677    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048686    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048691    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048696    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048701    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048709    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048713    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048720    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048725    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048730    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048734    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="util"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048739    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048744    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048750    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048755    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="pull"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048803    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e99c0895-6050-4811-bf21-509bec6bdfdc" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048809    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec98fb49-f08c-4a2d-b985-54f95f6110ad" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048818    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d53eb7a-257f-4cfd-af88-6165d8616467" containerName="extract"
Jul 06 14:32:29.048832 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.048824    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb488e0f-4ed2-4b35-9fae-40d3e88f3d78" containerName="extract"
Jul 06 14:32:29.051770 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.051755    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.054616 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.054593    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 14:32:29.054765 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.054634    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 14:32:29.054765 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.054671    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 14:32:29.054765 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.054735    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c89dr\""
Jul 06 14:32:29.062200 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.062182    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:32:29.080462 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.080421    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:32:29.113308 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.113275    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xkf\" (UniqueName: \"kubernetes.io/projected/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-kube-api-access-22xkf\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.113473 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.113340    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-config-file\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.214227 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.214190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-config-file\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.214399 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.214263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xkf\" (UniqueName: \"kubernetes.io/projected/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-kube-api-access-22xkf\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.214806 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.214787    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-config-file\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.226710 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.226678    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xkf\" (UniqueName: \"kubernetes.io/projected/dd8f3bea-92c0-4e31-b0fe-cb89de9b5246-kube-api-access-22xkf\") pod \"limitador-limitador-54d6d988d5-qg5ks\" (UID: \"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.362577 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.362541    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:29.499685 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.499656    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:32:29.501802 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:32:29.501778    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8f3bea_92c0_4e31_b0fe_cb89de9b5246.slice/crio-2102c069fe09b5b76a3bd5c671303a56a5a37b5f3afdb21e2c2f2156f38ba597 WatchSource:0}: Error finding container 2102c069fe09b5b76a3bd5c671303a56a5a37b5f3afdb21e2c2f2156f38ba597: Status 404 returned error can't find the container with id 2102c069fe09b5b76a3bd5c671303a56a5a37b5f3afdb21e2c2f2156f38ba597
Jul 06 14:32:29.505681 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.505660    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:29.510049 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.510033    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:29.512261 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.512236    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks" event={"ID":"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246","Type":"ContainerStarted","Data":"2102c069fe09b5b76a3bd5c671303a56a5a37b5f3afdb21e2c2f2156f38ba597"}
Jul 06 14:32:29.513399 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.513378    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6n5zb\""
Jul 06 14:32:29.516049 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.516029    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dg2q\" (UniqueName: \"kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q\") pod \"authorino-58cd955675-2st9m\" (UID: \"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47\") " pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:29.521877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.521854    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:29.616772 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.616667    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dg2q\" (UniqueName: \"kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q\") pod \"authorino-58cd955675-2st9m\" (UID: \"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47\") " pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:29.632102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.632077    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dg2q\" (UniqueName: \"kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q\") pod \"authorino-58cd955675-2st9m\" (UID: \"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47\") " pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:29.820745 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.820706    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:29.960112 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:29.960083    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:29.961958 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:32:29.961930    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1423477_4c2f_41ea_94dd_4c0a8b6c4d47.slice/crio-4e4558b3a70f82e7a70a24184bdb42be1b45118ed8d3eec34c96809fda298551 WatchSource:0}: Error finding container 4e4558b3a70f82e7a70a24184bdb42be1b45118ed8d3eec34c96809fda298551: Status 404 returned error can't find the container with id 4e4558b3a70f82e7a70a24184bdb42be1b45118ed8d3eec34c96809fda298551
Jul 06 14:32:30.519200 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:30.519167    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2st9m" event={"ID":"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47","Type":"ContainerStarted","Data":"4e4558b3a70f82e7a70a24184bdb42be1b45118ed8d3eec34c96809fda298551"}
Jul 06 14:32:33.896235 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:33.896202    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:34.546002 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.545948    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks" event={"ID":"dd8f3bea-92c0-4e31-b0fe-cb89de9b5246","Type":"ContainerStarted","Data":"e03e2831a9f1bb7d8ed5f771d5ae17b85ef33ef0d86312ebc73a5a092e9c3c74"}
Jul 06 14:32:34.546274 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.546033    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:32:34.547412 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.547386    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2st9m" event={"ID":"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47","Type":"ContainerStarted","Data":"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"}
Jul 06 14:32:34.547502 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.547460    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2st9m" podUID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" containerName="authorino" containerID="cri-o://e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9" gracePeriod=30
Jul 06 14:32:34.565561 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.565515    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks" podStartSLOduration=1.152165303 podStartE2EDuration="5.565498409s" totalImagesPullingTime="4.413333106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:34.564671741 +0000 UTC m=+504.773480158" watchObservedRunningTime="2026-07-06 14:32:34.565498409 +0000 UTC m=+504.774306795"
Jul 06 14:32:34.580138 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.580087    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2st9m" podStartSLOduration=1.620879839 podStartE2EDuration="5.580071285s" totalImagesPullingTime="3.959191446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:34.579574738 +0000 UTC m=+504.788383125" watchObservedRunningTime="2026-07-06 14:32:34.580071285 +0000 UTC m=+504.788879671"
Jul 06 14:32:34.805589 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.805521    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:34.861590 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.861550    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dg2q\" (UniqueName: \"kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q\") pod \"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47\" (UID: \"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47\") "
Jul 06 14:32:34.863612 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.863579    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q" (OuterVolumeSpecName: "kube-api-access-7dg2q") pod "e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" (UID: "e1423477-4c2f-41ea-94dd-4c0a8b6c4d47"). InnerVolumeSpecName "kube-api-access-7dg2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:32:34.962126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:34.962094    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dg2q\" (UniqueName: \"kubernetes.io/projected/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47-kube-api-access-7dg2q\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:32:35.554125 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.554091    2570 generic.go:356] "Generic (PLEG): container finished" podID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" containerID="e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9" exitCode=0
Jul 06 14:32:35.554303 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.554138    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2st9m"
Jul 06 14:32:35.554303 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.554179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2st9m" event={"ID":"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47","Type":"ContainerDied","Data":"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"}
Jul 06 14:32:35.554303 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.554220    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2st9m" event={"ID":"e1423477-4c2f-41ea-94dd-4c0a8b6c4d47","Type":"ContainerDied","Data":"4e4558b3a70f82e7a70a24184bdb42be1b45118ed8d3eec34c96809fda298551"}
Jul 06 14:32:35.554303 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.554241    2570 scope.go:117] "RemoveContainer" containerID="e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"
Jul 06 14:32:35.562836 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.562817    2570 scope.go:117] "RemoveContainer" containerID="e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"
Jul 06 14:32:35.563110 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:32:35.563092    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9\": container with ID starting with e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9 not found: ID does not exist" containerID="e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"
Jul 06 14:32:35.563166 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.563120    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9"} err="failed to get container status \"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9\": rpc error: code = NotFound desc = could not find container \"e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9\": container with ID starting with e3af3653410ef90bf026ae6064ce502553c2b4287e55f9c95cde5f6d941e41e9 not found: ID does not exist"
Jul 06 14:32:35.579193 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.579169    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:35.583374 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:35.583352    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2st9m"]
Jul 06 14:32:36.293093 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:36.293055    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" path="/var/lib/kubelet/pods/e1423477-4c2f-41ea-94dd-4c0a8b6c4d47/volumes"
Jul 06 14:32:45.555665 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:32:45.555633    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"
Jul 06 14:33:03.704490 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.704405    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:03.704985 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.704781    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" containerName="authorino"
Jul 06 14:33:03.704985 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.704796    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" containerName="authorino"
Jul 06 14:33:03.704985 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.704852    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1423477-4c2f-41ea-94dd-4c0a8b6c4d47" containerName="authorino"
Jul 06 14:33:03.706642 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.706627    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:03.709894 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.709875    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6n5zb\""
Jul 06 14:33:03.717300 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.717279    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:03.815618 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.815586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt2x8\" (UniqueName: \"kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8\") pod \"authorino-5d4b67c446-jmzl2\" (UID: \"57656d62-c9bd-418b-9696-c9c21927903b\") " pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:03.916057 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.916021    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt2x8\" (UniqueName: \"kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8\") pod \"authorino-5d4b67c446-jmzl2\" (UID: \"57656d62-c9bd-418b-9696-c9c21927903b\") " pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:03.926871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.926843    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt2x8\" (UniqueName: \"kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8\") pod \"authorino-5d4b67c446-jmzl2\" (UID: \"57656d62-c9bd-418b-9696-c9c21927903b\") " pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:03.943218 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.943188    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:03.943447 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:03.943435    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:04.083799 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.083774    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:04.086211 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:33:04.086184    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57656d62_c9bd_418b_9696_c9c21927903b.slice/crio-2f6d3f486f2086be55eef474d357c146fda8d57fa42641ae6db0e787c0f059a5 WatchSource:0}: Error finding container 2f6d3f486f2086be55eef474d357c146fda8d57fa42641ae6db0e787c0f059a5: Status 404 returned error can't find the container with id 2f6d3f486f2086be55eef474d357c146fda8d57fa42641ae6db0e787c0f059a5
Jul 06 14:33:04.429296 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.429257    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:33:04.432346 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.432321    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.435223 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.435198    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 14:33:04.441340 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.441318    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:33:04.521495 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.521460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5mb\" (UniqueName: \"kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.521495 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.521513    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.622455 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.622424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5mb\" (UniqueName: \"kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.622637 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.622468    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.625021 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.624979    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.632830 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.632795    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5mb\" (UniqueName: \"kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb\") pod \"authorino-5c77d7645b-zwmkq\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") " pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.696014 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.695953    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" event={"ID":"57656d62-c9bd-418b-9696-c9c21927903b","Type":"ContainerStarted","Data":"2f6d3f486f2086be55eef474d357c146fda8d57fa42641ae6db0e787c0f059a5"}
Jul 06 14:33:04.742810 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.742780    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:33:04.880126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:04.880101    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:33:04.881975 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:33:04.881949    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod873199bd_63c7_4264_9cc9_d4194e7e256e.slice/crio-93fac8d6be10844899439bed454a4b81c853bee1ac36ab7ccaeac3b81b4256c1 WatchSource:0}: Error finding container 93fac8d6be10844899439bed454a4b81c853bee1ac36ab7ccaeac3b81b4256c1: Status 404 returned error can't find the container with id 93fac8d6be10844899439bed454a4b81c853bee1ac36ab7ccaeac3b81b4256c1
Jul 06 14:33:05.703102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.703060    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" event={"ID":"873199bd-63c7-4264-9cc9-d4194e7e256e","Type":"ContainerStarted","Data":"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"}
Jul 06 14:33:05.703309 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.703111    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" event={"ID":"873199bd-63c7-4264-9cc9-d4194e7e256e","Type":"ContainerStarted","Data":"93fac8d6be10844899439bed454a4b81c853bee1ac36ab7ccaeac3b81b4256c1"}
Jul 06 14:33:05.704599 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.704573    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" event={"ID":"57656d62-c9bd-418b-9696-c9c21927903b","Type":"ContainerStarted","Data":"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"}
Jul 06 14:33:05.704700 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.704642    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" podUID="57656d62-c9bd-418b-9696-c9c21927903b" containerName="authorino" containerID="cri-o://32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a" gracePeriod=30
Jul 06 14:33:05.720819 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.720774    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" podStartSLOduration=1.298941225 podStartE2EDuration="1.720760649s" totalImagesPullingTime="421.819424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:05.719627154 +0000 UTC m=+535.928435539" watchObservedRunningTime="2026-07-06 14:33:05.720760649 +0000 UTC m=+535.929569035"
Jul 06 14:33:05.736871 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.736824    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" podStartSLOduration=2.160147005 podStartE2EDuration="2.736809562s" totalImagesPullingTime="576.662557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:05.735477336 +0000 UTC m=+535.944285722" watchObservedRunningTime="2026-07-06 14:33:05.736809562 +0000 UTC m=+535.945617947"
Jul 06 14:33:05.962479 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:05.962418    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:06.035472 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.035439    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt2x8\" (UniqueName: \"kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8\") pod \"57656d62-c9bd-418b-9696-c9c21927903b\" (UID: \"57656d62-c9bd-418b-9696-c9c21927903b\") "
Jul 06 14:33:06.037638 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.037610    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8" (OuterVolumeSpecName: "kube-api-access-qt2x8") pod "57656d62-c9bd-418b-9696-c9c21927903b" (UID: "57656d62-c9bd-418b-9696-c9c21927903b"). InnerVolumeSpecName "kube-api-access-qt2x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:33:06.137297 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.137233    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt2x8\" (UniqueName: \"kubernetes.io/projected/57656d62-c9bd-418b-9696-c9c21927903b-kube-api-access-qt2x8\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:33:06.710934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.710899    2570 generic.go:356] "Generic (PLEG): container finished" podID="57656d62-c9bd-418b-9696-c9c21927903b" containerID="32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a" exitCode=0
Jul 06 14:33:06.711108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.710959    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jmzl2"
Jul 06 14:33:06.711108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.711004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" event={"ID":"57656d62-c9bd-418b-9696-c9c21927903b","Type":"ContainerDied","Data":"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"}
Jul 06 14:33:06.711108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.711046    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jmzl2" event={"ID":"57656d62-c9bd-418b-9696-c9c21927903b","Type":"ContainerDied","Data":"2f6d3f486f2086be55eef474d357c146fda8d57fa42641ae6db0e787c0f059a5"}
Jul 06 14:33:06.711108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.711063    2570 scope.go:117] "RemoveContainer" containerID="32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"
Jul 06 14:33:06.719587 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.719570    2570 scope.go:117] "RemoveContainer" containerID="32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"
Jul 06 14:33:06.719863 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:33:06.719848    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a\": container with ID starting with 32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a not found: ID does not exist" containerID="32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"
Jul 06 14:33:06.719902 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.719871    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a"} err="failed to get container status \"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a\": rpc error: code = NotFound desc = could not find container \"32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a\": container with ID starting with 32f4dbd38817511a77468975eb2a4b67e31c1a63b980355a16f60ccc2ae1472a not found: ID does not exist"
Jul 06 14:33:06.728636 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.728606    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:06.732886 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:06.732863    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jmzl2"]
Jul 06 14:33:08.292183 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:08.292151    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57656d62-c9bd-418b-9696-c9c21927903b" path="/var/lib/kubelet/pods/57656d62-c9bd-418b-9696-c9c21927903b/volumes"
Jul 06 14:33:47.410216 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:33:47.410183    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:34:10.204422 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:10.204393    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:34:10.204901 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:10.204438    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:34:13.838039 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:13.838004    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:34:30.917496 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:30.917452    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:34:36.220034 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:36.219922    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:34:42.214171 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:42.214130    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:34:47.068243 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:34:47.068207    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:35:04.226439 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:35:04.226403    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:36:01.463315 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.463277    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-545756f545-qhhcq"]
Jul 06 14:36:01.463776 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.463584    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57656d62-c9bd-418b-9696-c9c21927903b" containerName="authorino"
Jul 06 14:36:01.463776 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.463594    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="57656d62-c9bd-418b-9696-c9c21927903b" containerName="authorino"
Jul 06 14:36:01.463776 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.463659    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="57656d62-c9bd-418b-9696-c9c21927903b" containerName="authorino"
Jul 06 14:36:01.466516 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.466489    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.539729 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.539697    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-545756f545-qhhcq"]
Jul 06 14:36:01.542867 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.542843    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bnj6\" (UniqueName: \"kubernetes.io/projected/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-kube-api-access-8bnj6\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.543017 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.542881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-tls-cert\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.644122 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.644089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bnj6\" (UniqueName: \"kubernetes.io/projected/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-kube-api-access-8bnj6\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.644122 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.644125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-tls-cert\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.646551 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.646533    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-tls-cert\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.664314 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.664288    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bnj6\" (UniqueName: \"kubernetes.io/projected/ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1-kube-api-access-8bnj6\") pod \"authorino-545756f545-qhhcq\" (UID: \"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1\") " pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.775536 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.775448    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-545756f545-qhhcq"
Jul 06 14:36:01.953007 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.952963    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-545756f545-qhhcq"]
Jul 06 14:36:01.959923 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:36:01.959894    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podceea96f3_4ef8_4fcb_a7f5_00f36a5f5ed1.slice/crio-1d384b14a15e88652838a390dcc6450b6f139ddc1ffe1ba79c231b7a81187d5f WatchSource:0}: Error finding container 1d384b14a15e88652838a390dcc6450b6f139ddc1ffe1ba79c231b7a81187d5f: Status 404 returned error can't find the container with id 1d384b14a15e88652838a390dcc6450b6f139ddc1ffe1ba79c231b7a81187d5f
Jul 06 14:36:01.961673 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:01.961658    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:36:02.593932 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.593887    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-545756f545-qhhcq" event={"ID":"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1","Type":"ContainerStarted","Data":"b8517f9159bb4d6b980d97a690e4eb90337e3c16ab1d7fdbcff9987f8680ebd9"}
Jul 06 14:36:02.593932 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.593933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-545756f545-qhhcq" event={"ID":"ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1","Type":"ContainerStarted","Data":"1d384b14a15e88652838a390dcc6450b6f139ddc1ffe1ba79c231b7a81187d5f"}
Jul 06 14:36:02.627447 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.627348    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-545756f545-qhhcq" podStartSLOduration=1.221344327 podStartE2EDuration="1.627332086s" totalImagesPullingTime="405.987759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:36:02.627265698 +0000 UTC m=+712.836074086" watchObservedRunningTime="2026-07-06 14:36:02.627332086 +0000 UTC m=+712.836140449"
Jul 06 14:36:02.694389 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.694355    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:36:02.694574 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.694534    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" podUID="873199bd-63c7-4264-9cc9-d4194e7e256e" containerName="authorino" containerID="cri-o://5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40" gracePeriod=30
Jul 06 14:36:02.958503 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:02.958476    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:36:03.055185 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.055152    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert\") pod \"873199bd-63c7-4264-9cc9-d4194e7e256e\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") "
Jul 06 14:36:03.055374 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.055213    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq5mb\" (UniqueName: \"kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb\") pod \"873199bd-63c7-4264-9cc9-d4194e7e256e\" (UID: \"873199bd-63c7-4264-9cc9-d4194e7e256e\") "
Jul 06 14:36:03.057233 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.057194    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb" (OuterVolumeSpecName: "kube-api-access-sq5mb") pod "873199bd-63c7-4264-9cc9-d4194e7e256e" (UID: "873199bd-63c7-4264-9cc9-d4194e7e256e"). InnerVolumeSpecName "kube-api-access-sq5mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:36:03.074600 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.074558    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "873199bd-63c7-4264-9cc9-d4194e7e256e" (UID: "873199bd-63c7-4264-9cc9-d4194e7e256e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:36:03.155962 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.155926    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/873199bd-63c7-4264-9cc9-d4194e7e256e-tls-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:36:03.155962 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.155959    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq5mb\" (UniqueName: \"kubernetes.io/projected/873199bd-63c7-4264-9cc9-d4194e7e256e-kube-api-access-sq5mb\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:36:03.600250 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.600213    2570 generic.go:356] "Generic (PLEG): container finished" podID="873199bd-63c7-4264-9cc9-d4194e7e256e" containerID="5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40" exitCode=0
Jul 06 14:36:03.600698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.600254    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77d7645b-zwmkq"
Jul 06 14:36:03.600698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.600292    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" event={"ID":"873199bd-63c7-4264-9cc9-d4194e7e256e","Type":"ContainerDied","Data":"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"}
Jul 06 14:36:03.600698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.600339    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77d7645b-zwmkq" event={"ID":"873199bd-63c7-4264-9cc9-d4194e7e256e","Type":"ContainerDied","Data":"93fac8d6be10844899439bed454a4b81c853bee1ac36ab7ccaeac3b81b4256c1"}
Jul 06 14:36:03.600698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.600382    2570 scope.go:117] "RemoveContainer" containerID="5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"
Jul 06 14:36:03.615080 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.615054    2570 scope.go:117] "RemoveContainer" containerID="5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"
Jul 06 14:36:03.615393 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:36:03.615373    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40\": container with ID starting with 5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40 not found: ID does not exist" containerID="5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"
Jul 06 14:36:03.615468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.615407    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40"} err="failed to get container status \"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40\": rpc error: code = NotFound desc = could not find container \"5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40\": container with ID starting with 5cd7cba8624533155bd33c77cfcc5b8c239a477a83677f34107ca23a4ac0de40 not found: ID does not exist"
Jul 06 14:36:03.634588 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.634547    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:36:03.651088 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:03.651047    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c77d7645b-zwmkq"]
Jul 06 14:36:04.296381 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:04.296332    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="873199bd-63c7-4264-9cc9-d4194e7e256e" path="/var/lib/kubelet/pods/873199bd-63c7-4264-9cc9-d4194e7e256e/volumes"
Jul 06 14:36:31.031185 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:31.031147    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:36:41.651723 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:41.651690    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:36:53.132439 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:53.132400    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:36:58.822974 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:36:58.822938    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:37:09.123668 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:37:09.123625    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:37:16.530530 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:37:16.530485    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:37:39.276493 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:37:39.276402    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:37:49.546427 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:37:49.546390    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:38:27.836450 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:38:27.836412    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:38:37.663680 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:38:37.663645    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:38:46.891637 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:38:46.891597    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:38:55.556073 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:38:55.555931    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:39:04.446652 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:04.446564    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:39:10.239707 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:10.239678    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:39:10.243043 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:10.243015    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:39:15.322240 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:15.322205    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:39:26.730304 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:26.730269    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:39:36.224648 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:36.224608    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:39:59.341421 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:39:59.341376    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:40:09.119430 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:40:09.119395    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:40:35.116433 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:40:35.116335    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:40:47.040860 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:40:47.040827    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:01.820478 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:01.820439    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:11.019471 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:11.019424    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:22.029833 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:22.029799    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:31.113520 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:31.113482    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:39.518078 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:39.518035    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:41:55.423636 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.423595    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-647d98b875-85hnl"]
Jul 06 14:41:55.424691 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.423919    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873199bd-63c7-4264-9cc9-d4194e7e256e" containerName="authorino"
Jul 06 14:41:55.424691 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.423930    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="873199bd-63c7-4264-9cc9-d4194e7e256e" containerName="authorino"
Jul 06 14:41:55.424691 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.423979    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="873199bd-63c7-4264-9cc9-d4194e7e256e" containerName="authorino"
Jul 06 14:41:55.427346 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.427319    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.449645 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.449613    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-647d98b875-85hnl"]
Jul 06 14:41:55.539649 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539614    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-console-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.539649 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539651    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.539870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-oauth-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.539870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539703    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-oauth-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.539870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkdbr\" (UniqueName: \"kubernetes.io/projected/f780604b-3155-425a-aec0-abfb026dd718-kube-api-access-wkdbr\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.539870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-trusted-ca-bundle\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.540038 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.539897    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-service-ca\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.640743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-console-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.640743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640770    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-oauth-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640818    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-oauth-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640850    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkdbr\" (UniqueName: \"kubernetes.io/projected/f780604b-3155-425a-aec0-abfb026dd718-kube-api-access-wkdbr\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640880    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-trusted-ca-bundle\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641033 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.640939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-service-ca\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641703 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.641676    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-service-ca\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641855 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.641836    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-console-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.641983 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.641953    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-oauth-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.642077 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.642062    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f780604b-3155-425a-aec0-abfb026dd718-trusted-ca-bundle\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.643716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.643689    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-oauth-config\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.644274 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.644249    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f780604b-3155-425a-aec0-abfb026dd718-console-serving-cert\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.650876 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.650851    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkdbr\" (UniqueName: \"kubernetes.io/projected/f780604b-3155-425a-aec0-abfb026dd718-kube-api-access-wkdbr\") pod \"console-647d98b875-85hnl\" (UID: \"f780604b-3155-425a-aec0-abfb026dd718\") " pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.737757 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.737652    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:41:55.886137 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:55.883364    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-647d98b875-85hnl"]
Jul 06 14:41:55.887743 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:41:55.887708    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf780604b_3155_425a_aec0_abfb026dd718.slice/crio-e05da5834181e2ab484b33dffae71f4c5ead5309b57c047db34225a97f1d0769 WatchSource:0}: Error finding container e05da5834181e2ab484b33dffae71f4c5ead5309b57c047db34225a97f1d0769: Status 404 returned error can't find the container with id e05da5834181e2ab484b33dffae71f4c5ead5309b57c047db34225a97f1d0769
Jul 06 14:41:56.373920 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:56.373881    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647d98b875-85hnl" event={"ID":"f780604b-3155-425a-aec0-abfb026dd718","Type":"ContainerStarted","Data":"f779f8e6761e0b1cee7fdafd645695fd266117a7c5650542e9140385f19f8491"}
Jul 06 14:41:56.374149 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:56.373938    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647d98b875-85hnl" event={"ID":"f780604b-3155-425a-aec0-abfb026dd718","Type":"ContainerStarted","Data":"e05da5834181e2ab484b33dffae71f4c5ead5309b57c047db34225a97f1d0769"}
Jul 06 14:41:56.399752 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:41:56.399702    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-647d98b875-85hnl" podStartSLOduration=1.399688526 podStartE2EDuration="1.399688526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:41:56.396737918 +0000 UTC m=+1066.605546330" watchObservedRunningTime="2026-07-06 14:41:56.399688526 +0000 UTC m=+1066.608496911"
Jul 06 14:42:05.738726 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:05.738639    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:42:05.738726 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:05.738695    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:42:05.744421 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:05.744396    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:42:06.427809 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:06.427781    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-647d98b875-85hnl"
Jul 06 14:42:06.481973 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:06.481937    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:42:31.504326 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.504288    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-684c4dd45d-lrkn8" podUID="1c2539e6-2526-4671-9410-4a3fe7db3c26" containerName="console" containerID="cri-o://a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f" gracePeriod=15
Jul 06 14:42:31.753449 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.753405    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:42:31.845435 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845399    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845608 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845453    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845608 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845495    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845608 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845548    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845608 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845573    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r44qt\" (UniqueName: \"kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845827 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845620    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845827 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845662    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config\") pod \"1c2539e6-2526-4671-9410-4a3fe7db3c26\" (UID: \"1c2539e6-2526-4671-9410-4a3fe7db3c26\") "
Jul 06 14:42:31.845955 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845924    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:42:31.846049 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845946    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config" (OuterVolumeSpecName: "console-config") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:42:31.846049 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.845964    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:42:31.846153 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.846051    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:42:31.847755 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.847729    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:42:31.847857 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.847771    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:42:31.847909 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.847865    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt" (OuterVolumeSpecName: "kube-api-access-r44qt") pod "1c2539e6-2526-4671-9410-4a3fe7db3c26" (UID: "1c2539e6-2526-4671-9410-4a3fe7db3c26"). InnerVolumeSpecName "kube-api-access-r44qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:42:31.946820 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946777    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.946820 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946814    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r44qt\" (UniqueName: \"kubernetes.io/projected/1c2539e6-2526-4671-9410-4a3fe7db3c26-kube-api-access-r44qt\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.946820 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946825    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-service-ca\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.947138 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946834    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-oauth-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.947138 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946843    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-trusted-ca-bundle\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.947138 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946852    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-oauth-serving-cert\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:31.947138 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:31.946860    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2539e6-2526-4671-9410-4a3fe7db3c26-console-config\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:42:32.547350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.547319    2570 generic.go:356] "Generic (PLEG): container finished" podID="1c2539e6-2526-4671-9410-4a3fe7db3c26" containerID="a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f" exitCode=0
Jul 06 14:42:32.547735 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.547363    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684c4dd45d-lrkn8"
Jul 06 14:42:32.547735 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.547402    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684c4dd45d-lrkn8" event={"ID":"1c2539e6-2526-4671-9410-4a3fe7db3c26","Type":"ContainerDied","Data":"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"}
Jul 06 14:42:32.547735 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.547449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684c4dd45d-lrkn8" event={"ID":"1c2539e6-2526-4671-9410-4a3fe7db3c26","Type":"ContainerDied","Data":"13bfc7987018ff727604da91f0b9b5f8d5d18fcd4f149f22728d8d35c26b4fba"}
Jul 06 14:42:32.547735 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.547474    2570 scope.go:117] "RemoveContainer" containerID="a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"
Jul 06 14:42:32.556304 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.556277    2570 scope.go:117] "RemoveContainer" containerID="a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"
Jul 06 14:42:32.556568 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:42:32.556548    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f\": container with ID starting with a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f not found: ID does not exist" containerID="a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"
Jul 06 14:42:32.556615 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.556576    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f"} err="failed to get container status \"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f\": rpc error: code = NotFound desc = could not find container \"a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f\": container with ID starting with a8d502f701615e1d779ffd1be09adc2c457283bc03fbfd511f4481e9afe4021f not found: ID does not exist"
Jul 06 14:42:32.567673 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.567651    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:42:32.572141 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:32.572121    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-684c4dd45d-lrkn8"]
Jul 06 14:42:34.292653 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:42:34.292617    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2539e6-2526-4671-9410-4a3fe7db3c26" path="/var/lib/kubelet/pods/1c2539e6-2526-4671-9410-4a3fe7db3c26/volumes"
Jul 06 14:43:27.629540 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:43:27.629497    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:43:39.027826 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:43:39.027790    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:43:49.329220 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:43:49.329176    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:00.924978 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:00.924940    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:10.275613 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:10.275581    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:44:10.278940 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:10.278916    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:44:12.843126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:12.843095    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:21.626139 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:21.626103    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:31.331772 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:31.331736    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:42.013105 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:42.013069    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:44:53.291393 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:44:53.291359    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:45:02.635661 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:45:02.635580    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:45:12.720211 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:45:12.720176    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:45:37.735350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:45:37.735311    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:45:45.132173 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:45:45.132142    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:45:54.831713 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:45:54.831675    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:06.616126 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:06.616093    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:15.117132 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:15.117098    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:22.726827 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:22.726786    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:31.934675 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:31.934594    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:40.719025 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:40.718970    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:50.526241 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:50.526205    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:46:59.318327 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:46:59.318292    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:47:08.415388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:47:08.415346    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:47:21.425418 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:47:21.425387    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:47:32.517181 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:47:32.517138    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:47:41.622315 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:47:41.622279    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:47:51.205578 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:47:51.205541    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:04.412077 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:04.411972    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:12.808507 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:12.808470    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:23.715509 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:23.715476    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:34.814060 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:34.814010    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:46.812960 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:46.812924    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:48:53.609172 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:48:53.609139    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:49:09.407571 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:49:09.407536    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:49:10.313213 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:49:10.313185    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:49:10.316433 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:49:10.316410    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:49:19.417320 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:49:19.417290    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:51:20.222966 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:51:20.222928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:51:29.139102 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:51:29.139066    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:51:40.717017 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:51:40.716973    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:51:52.922728 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:51:52.922695    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:52:25.816144 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:25.816109    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:52:47.428907 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.428873    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:52:47.429314 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.429202    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2539e6-2526-4671-9410-4a3fe7db3c26" containerName="console"
Jul 06 14:52:47.429314 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.429213    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2539e6-2526-4671-9410-4a3fe7db3c26" containerName="console"
Jul 06 14:52:47.429314 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.429271    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2539e6-2526-4671-9410-4a3fe7db3c26" containerName="console"
Jul 06 14:52:47.431999 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.431970    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.436460 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.436433    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-277e79e2\"/\"openshift-service-ca.crt\""
Jul 06 14:52:47.436599 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.436434    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-277e79e2\"/\"e2e-lifecycle-model-277e79e2-kserve-self-signed-certs\""
Jul 06 14:52:47.436599 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.436522    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-277e79e2\"/\"kube-root-ca.crt\""
Jul 06 14:52:47.436599 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.436442    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-277e79e2\"/\"default-dockercfg-dhfp9\""
Jul 06 14:52:47.447681 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.447650    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:52:47.551587 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.551764 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdctd\" (UniqueName: \"kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.551764 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.551843 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551762    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.551843 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551785    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.551843 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.551812    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652423 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652377    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652423 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652460    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652493    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652697 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652540    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdctd\" (UniqueName: \"kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652858 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652720    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652913 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652879    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.652980 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.652908    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.653074 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.653057    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.654586 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.654568    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.654931 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.654912    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.662958 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.662938    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdctd\" (UniqueName: \"kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd\") pod \"e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") " pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.741596 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.741484    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:47.891921 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.891897    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:52:47.894076 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:52:47.894042    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43216eb4_8867_4c86_aa45_e52a9ab64de7.slice/crio-68b4188838e5918dbee881e0366a9fcbefa07e23241c2a97ace57a9f02d89115 WatchSource:0}: Error finding container 68b4188838e5918dbee881e0366a9fcbefa07e23241c2a97ace57a9f02d89115: Status 404 returned error can't find the container with id 68b4188838e5918dbee881e0366a9fcbefa07e23241c2a97ace57a9f02d89115
Jul 06 14:52:47.895877 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:47.895858    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:52:48.596504 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:48.596462    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" event={"ID":"43216eb4-8867-4c86-aa45-e52a9ab64de7","Type":"ContainerStarted","Data":"68b4188838e5918dbee881e0366a9fcbefa07e23241c2a97ace57a9f02d89115"}
Jul 06 14:52:49.603456 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:49.603421    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" event={"ID":"43216eb4-8867-4c86-aa45-e52a9ab64de7","Type":"ContainerStarted","Data":"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"}
Jul 06 14:52:49.640051 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:49.639974    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" podStartSLOduration=1.485236919 podStartE2EDuration="2.63995713s" totalImagesPullingTime="1.154720211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:52:49.638334405 +0000 UTC m=+1719.847142793" watchObservedRunningTime="2026-07-06 14:52:49.63995713 +0000 UTC m=+1719.848765515"
Jul 06 14:52:50.533364 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:50.533329    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:52:57.742464 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:57.742429    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:57.743112 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:57.742511    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:57.754783 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:57.754763    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:52:58.658164 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:52:58.658131    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:53:07.423774 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:07.423733    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:53:17.017970 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:17.017938    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:53:24.179826 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:24.179788    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:53:28.513086 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:28.513046    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:53:39.200070 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.199970    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" podUID="43216eb4-8867-4c86-aa45-e52a9ab64de7" containerName="main" containerID="cri-o://a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46" gracePeriod=45
Jul 06 14:53:39.450114 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.450055    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:53:39.598541 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598510    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.598736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598551    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdctd\" (UniqueName: \"kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.598736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598572    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.598736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598598    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.598736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598630    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.598736 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598686    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home\") pod \"43216eb4-8867-4c86-aa45-e52a9ab64de7\" (UID: \"43216eb4-8867-4c86-aa45-e52a9ab64de7\") "
Jul 06 14:53:39.599028 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598955    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache" (OuterVolumeSpecName: "model-cache") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:39.599134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.598974    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:39.599134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.599046    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home" (OuterVolumeSpecName: "home") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:39.600848 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.600825    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm" (OuterVolumeSpecName: "dshm") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:39.600971 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.600939    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd" (OuterVolumeSpecName: "kube-api-access-jdctd") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "kube-api-access-jdctd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:53:39.600971 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.600951    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43216eb4-8867-4c86-aa45-e52a9ab64de7" (UID: "43216eb4-8867-4c86-aa45-e52a9ab64de7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:53:39.699686 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699649    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43216eb4-8867-4c86-aa45-e52a9ab64de7-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.699686 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699682    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdctd\" (UniqueName: \"kubernetes.io/projected/43216eb4-8867-4c86-aa45-e52a9ab64de7-kube-api-access-jdctd\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.699686 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699691    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.700186 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699700    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.700186 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699709    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.700186 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.699716    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43216eb4-8867-4c86-aa45-e52a9ab64de7-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:53:39.852213 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.852181    2570 generic.go:356] "Generic (PLEG): container finished" podID="43216eb4-8867-4c86-aa45-e52a9ab64de7" containerID="a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46" exitCode=0
Jul 06 14:53:39.852388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.852226    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"
Jul 06 14:53:39.852388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.852250    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" event={"ID":"43216eb4-8867-4c86-aa45-e52a9ab64de7","Type":"ContainerDied","Data":"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"}
Jul 06 14:53:39.852388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.852279    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g" event={"ID":"43216eb4-8867-4c86-aa45-e52a9ab64de7","Type":"ContainerDied","Data":"68b4188838e5918dbee881e0366a9fcbefa07e23241c2a97ace57a9f02d89115"}
Jul 06 14:53:39.852388 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.852296    2570 scope.go:117] "RemoveContainer" containerID="a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"
Jul 06 14:53:39.861554 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.861535    2570 scope.go:117] "RemoveContainer" containerID="a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"
Jul 06 14:53:39.861837 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:53:39.861814    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46\": container with ID starting with a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46 not found: ID does not exist" containerID="a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"
Jul 06 14:53:39.861888 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.861850    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46"} err="failed to get container status \"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46\": rpc error: code = NotFound desc = could not find container \"a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46\": container with ID starting with a1db17bb999412128743fbd3a97aa2de84e3672bcf88702512bd6c36cf8d1f46 not found: ID does not exist"
Jul 06 14:53:39.875137 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.875112    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:53:39.878703 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:39.878676    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-277e79e2/e2e-lifecycle-model-277e79e2-kserve-64cdbd9c9f-t2s8g"]
Jul 06 14:53:40.292624 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:53:40.292591    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43216eb4-8867-4c86-aa45-e52a9ab64de7" path="/var/lib/kubelet/pods/43216eb4-8867-4c86-aa45-e52a9ab64de7/volumes"
Jul 06 14:54:04.610415 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:54:04.610338    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:54:10.352675 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:54:10.352648    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:54:10.355198 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:54:10.355176    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:54:27.512614 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:54:27.512569    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:54:37.910599 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:54:37.910563    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:55:30.220140 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:55:30.220102    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:55:44.507415 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:55:44.507381    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:56:03.503441 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.503403    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:56:03.503969 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.503847    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43216eb4-8867-4c86-aa45-e52a9ab64de7" containerName="main"
Jul 06 14:56:03.503969 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.503860    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="43216eb4-8867-4c86-aa45-e52a9ab64de7" containerName="main"
Jul 06 14:56:03.503969 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.503931    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="43216eb4-8867-4c86-aa45-e52a9ab64de7" containerName="main"
Jul 06 14:56:03.506848 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.506833    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.510626 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.510599    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8e878a\"/\"kube-root-ca.crt\""
Jul 06 14:56:03.512495 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.512473    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8e878a\"/\"openshift-service-ca.crt\""
Jul 06 14:56:03.512495 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.512487    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8e878a\"/\"default-dockercfg-vvqr6\""
Jul 06 14:56:03.512735 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.512722    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8e878a\"/\"test-model-8e878a-kserve-self-signed-certs\""
Jul 06 14:56:03.518153 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.518134    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:56:03.599763 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599732    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.599927 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlqt6\" (UniqueName: \"kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.599927 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599862    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.599927 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599910    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.599927 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599928    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.600134 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.599980    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701301 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701268    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701301 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701305    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701569 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701569 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701380    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701569 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701417    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlqt6\" (UniqueName: \"kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701569 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701462    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701722    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701759    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.701887 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.701805    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.703722 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.703698    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.703957 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.703943    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.711781 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.711757    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlqt6\" (UniqueName: \"kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6\") pod \"test-model-8e878a-kserve-5ff88f58b7-s9rwl\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") " pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.815979 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.815883    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:03.952336 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:03.952307    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:56:03.954188 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:56:03.954163    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeafa95bf_4123_4b2d_a43c_079ad2149673.slice/crio-a5b048a4547822fc6acd9c0587b684551127728baf71431803946887a49ab124 WatchSource:0}: Error finding container a5b048a4547822fc6acd9c0587b684551127728baf71431803946887a49ab124: Status 404 returned error can't find the container with id a5b048a4547822fc6acd9c0587b684551127728baf71431803946887a49ab124
Jul 06 14:56:04.563775 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:04.563740    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" event={"ID":"eafa95bf-4123-4b2d-a43c-079ad2149673","Type":"ContainerStarted","Data":"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"}
Jul 06 14:56:04.563775 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:04.563781    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" event={"ID":"eafa95bf-4123-4b2d-a43c-079ad2149673","Type":"ContainerStarted","Data":"a5b048a4547822fc6acd9c0587b684551127728baf71431803946887a49ab124"}
Jul 06 14:56:04.584158 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:04.584113    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" podStartSLOduration=1.584097452 podStartE2EDuration="1.584097452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:56:04.582438391 +0000 UTC m=+1914.791246780" watchObservedRunningTime="2026-07-06 14:56:04.584097452 +0000 UTC m=+1914.792905838"
Jul 06 14:56:06.923635 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:06.923592    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:56:13.816225 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:13.816180    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:13.816723 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:13.816245    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:13.828966 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:13.828936    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:14.632073 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:14.632043    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:56:16.915357 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:16.915322    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:56:46.695468 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.695435    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:56:46.698722 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.698706    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.702815 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.702776    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5e9a43\"/\"openshift-service-ca.crt\""
Jul 06 14:56:46.702815 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.702776    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5e9a43\"/\"test-model-5e9a43-kserve-self-signed-certs\""
Jul 06 14:56:46.703024 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.702825    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5e9a43\"/\"kube-root-ca.crt\""
Jul 06 14:56:46.703024 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.702882    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5e9a43\"/\"default-dockercfg-s2gwk\""
Jul 06 14:56:46.707427 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.707404    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:56:46.761396 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761364    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.761396 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.761586 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.761586 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761524    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.761586 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761567    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.761688 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.761589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghpwg\" (UniqueName: \"kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.862888 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.862847    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863103 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.862896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghpwg\" (UniqueName: \"kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863103 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.862949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863103 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.862982    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863103 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.863044    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863302 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.863104    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863399 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.863381    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863464 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.863431    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.863523 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.863501    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.865359 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.865337    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.865491 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.865473    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:46.871758 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:46.871733    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghpwg\" (UniqueName: \"kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg\") pod \"test-model-5e9a43-kserve-5f4888d475-xqfw9\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") " pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:47.009737 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:47.009634    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:47.147019 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:47.146937    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:56:47.149263 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:56:47.149232    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0b010b5_6e30_4527_bbd0_663b661a9129.slice/crio-e9af8d7ef56b34ac26cbb81ccb66b8280ef5b4044150c90c6a63c9a85c1a98f9 WatchSource:0}: Error finding container e9af8d7ef56b34ac26cbb81ccb66b8280ef5b4044150c90c6a63c9a85c1a98f9: Status 404 returned error can't find the container with id e9af8d7ef56b34ac26cbb81ccb66b8280ef5b4044150c90c6a63c9a85c1a98f9
Jul 06 14:56:47.783977 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:47.783935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" event={"ID":"d0b010b5-6e30-4527-bbd0-663b661a9129","Type":"ContainerStarted","Data":"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"}
Jul 06 14:56:47.783977 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:47.783977    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" event={"ID":"d0b010b5-6e30-4527-bbd0-663b661a9129","Type":"ContainerStarted","Data":"e9af8d7ef56b34ac26cbb81ccb66b8280ef5b4044150c90c6a63c9a85c1a98f9"}
Jul 06 14:56:47.813095 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:47.812978    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" podStartSLOduration=1.812962095 podStartE2EDuration="1.812962095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:56:47.811717832 +0000 UTC m=+1958.020526219" watchObservedRunningTime="2026-07-06 14:56:47.812962095 +0000 UTC m=+1958.021770480"
Jul 06 14:56:51.624726 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:51.624693    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:56:57.010304 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:57.010266    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:57.010716 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:57.010320    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:57.022790 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:57.022765    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:56:57.849514 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:56:57.849485    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:57:15.240874 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:15.240840    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:57:16.212014 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:16.211969    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:57:19.828791 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:19.828756    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:57:30.261359 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.261309    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" podUID="eafa95bf-4123-4b2d-a43c-079ad2149673" containerName="main" containerID="cri-o://c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd" gracePeriod=45
Jul 06 14:57:30.514386 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.514329    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:57:30.638421 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638389    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638458    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638487    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlqt6\" (UniqueName: \"kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638507    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638533    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638564    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir\") pod \"eafa95bf-4123-4b2d-a43c-079ad2149673\" (UID: \"eafa95bf-4123-4b2d-a43c-079ad2149673\") "
Jul 06 14:57:30.638869 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638707    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home" (OuterVolumeSpecName: "home") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:30.638869 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638823    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:30.638869 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638803    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache" (OuterVolumeSpecName: "model-cache") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:30.639046 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.638865    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:30.640698 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.640671    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm" (OuterVolumeSpecName: "dshm") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:30.640793 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.640744    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:57:30.640793 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.640749    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6" (OuterVolumeSpecName: "kube-api-access-qlqt6") pod "eafa95bf-4123-4b2d-a43c-079ad2149673" (UID: "eafa95bf-4123-4b2d-a43c-079ad2149673"). InnerVolumeSpecName "kube-api-access-qlqt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:57:30.739282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.739242    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eafa95bf-4123-4b2d-a43c-079ad2149673-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:30.739282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.739273    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:30.739282 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.739285    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:30.739545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.739299    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlqt6\" (UniqueName: \"kubernetes.io/projected/eafa95bf-4123-4b2d-a43c-079ad2149673-kube-api-access-qlqt6\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:30.739545 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:30.739313    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eafa95bf-4123-4b2d-a43c-079ad2149673-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.012037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.011986    2570 generic.go:356] "Generic (PLEG): container finished" podID="eafa95bf-4123-4b2d-a43c-079ad2149673" containerID="c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd" exitCode=0
Jul 06 14:57:31.012037 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.012021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" event={"ID":"eafa95bf-4123-4b2d-a43c-079ad2149673","Type":"ContainerDied","Data":"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"}
Jul 06 14:57:31.012326 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.012051    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"
Jul 06 14:57:31.012326 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.012076    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl" event={"ID":"eafa95bf-4123-4b2d-a43c-079ad2149673","Type":"ContainerDied","Data":"a5b048a4547822fc6acd9c0587b684551127728baf71431803946887a49ab124"}
Jul 06 14:57:31.012326 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.012112    2570 scope.go:117] "RemoveContainer" containerID="c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"
Jul 06 14:57:31.021464 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.021445    2570 scope.go:117] "RemoveContainer" containerID="c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"
Jul 06 14:57:31.021733 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:57:31.021716    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd\": container with ID starting with c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd not found: ID does not exist" containerID="c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"
Jul 06 14:57:31.021809 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.021740    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd"} err="failed to get container status \"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd\": rpc error: code = NotFound desc = could not find container \"c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd\": container with ID starting with c60a8a04f8ffa0911b612aeae802c8f4d6779b811c9e205e28fd221ce183cebd not found: ID does not exist"
Jul 06 14:57:31.045466 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.045430    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:57:31.050328 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.050300    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8e878a/test-model-8e878a-kserve-5ff88f58b7-s9rwl"]
Jul 06 14:57:31.231671 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.231631    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" podUID="d0b010b5-6e30-4527-bbd0-663b661a9129" containerName="main" containerID="cri-o://e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2" gracePeriod=45
Jul 06 14:57:31.480192 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.480169    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:57:31.647188 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647154    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647368 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647199    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647368 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647249    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghpwg\" (UniqueName: \"kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647368 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647296    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647368 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647329    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647583 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647400    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir\") pod \"d0b010b5-6e30-4527-bbd0-663b661a9129\" (UID: \"d0b010b5-6e30-4527-bbd0-663b661a9129\") "
Jul 06 14:57:31.647583 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647483    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home" (OuterVolumeSpecName: "home") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:31.647583 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647547    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:31.647747 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647666    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:31.647747 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647719    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.647747 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.647737    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.649610 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.649579    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm" (OuterVolumeSpecName: "dshm") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:57:31.649610 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.649604    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:57:31.649746 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.649590    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg" (OuterVolumeSpecName: "kube-api-access-ghpwg") pod "d0b010b5-6e30-4527-bbd0-663b661a9129" (UID: "d0b010b5-6e30-4527-bbd0-663b661a9129"). InnerVolumeSpecName "kube-api-access-ghpwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:57:31.748353 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.748316    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghpwg\" (UniqueName: \"kubernetes.io/projected/d0b010b5-6e30-4527-bbd0-663b661a9129-kube-api-access-ghpwg\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.748353 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.748349    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b010b5-6e30-4527-bbd0-663b661a9129-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.748353 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.748360    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:31.748591 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:31.748367    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b010b5-6e30-4527-bbd0-663b661a9129-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 14:57:32.019554 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.019473    2570 generic.go:356] "Generic (PLEG): container finished" podID="d0b010b5-6e30-4527-bbd0-663b661a9129" containerID="e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2" exitCode=0
Jul 06 14:57:32.019554 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.019518    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"
Jul 06 14:57:32.019743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.019555    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" event={"ID":"d0b010b5-6e30-4527-bbd0-663b661a9129","Type":"ContainerDied","Data":"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"}
Jul 06 14:57:32.019743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.019594    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9" event={"ID":"d0b010b5-6e30-4527-bbd0-663b661a9129","Type":"ContainerDied","Data":"e9af8d7ef56b34ac26cbb81ccb66b8280ef5b4044150c90c6a63c9a85c1a98f9"}
Jul 06 14:57:32.019743 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.019615    2570 scope.go:117] "RemoveContainer" containerID="e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"
Jul 06 14:57:32.029035 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.029013    2570 scope.go:117] "RemoveContainer" containerID="e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"
Jul 06 14:57:32.029270 ip-10-0-140-204 kubenswrapper[2570]: E0706 14:57:32.029254    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2\": container with ID starting with e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2 not found: ID does not exist" containerID="e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"
Jul 06 14:57:32.029315 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.029279    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2"} err="failed to get container status \"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2\": rpc error: code = NotFound desc = could not find container \"e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2\": container with ID starting with e1ed4084fafa1feb4274aed0bbb20ff91dd5d03dc461eefd9f69ef686da2b1c2 not found: ID does not exist"
Jul 06 14:57:32.045083 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.045046    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:57:32.046952 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.046932    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5e9a43/test-model-5e9a43-kserve-5f4888d475-xqfw9"]
Jul 06 14:57:32.292742 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.292664    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b010b5-6e30-4527-bbd0-663b661a9129" path="/var/lib/kubelet/pods/d0b010b5-6e30-4527-bbd0-663b661a9129/volumes"
Jul 06 14:57:32.293071 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:57:32.293057    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eafa95bf-4123-4b2d-a43c-079ad2149673" path="/var/lib/kubelet/pods/eafa95bf-4123-4b2d-a43c-079ad2149673/volumes"
Jul 06 14:58:04.823502 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:58:04.823467    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:58:17.977787 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:58:17.977751    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:58:32.919771 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:58:32.919677    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:58:44.218048 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:58:44.218011    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:58:58.014202 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:58:58.014164    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:59:00.008859 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.008821    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009159    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafa95bf-4123-4b2d-a43c-079ad2149673" containerName="main"
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009171    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafa95bf-4123-4b2d-a43c-079ad2149673" containerName="main"
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009193    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b010b5-6e30-4527-bbd0-663b661a9129" containerName="main"
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009198    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b010b5-6e30-4527-bbd0-663b661a9129" containerName="main"
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009246    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafa95bf-4123-4b2d-a43c-079ad2149673" containerName="main"
Jul 06 14:59:00.009369 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.009255    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b010b5-6e30-4527-bbd0-663b661a9129" containerName="main"
Jul 06 14:59:00.012422 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.012401    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.015350 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.015329    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-06dd21-serving-cert\""
Jul 06 14:59:00.015485 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.015400    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jt75v\""
Jul 06 14:59:00.022798 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.022777    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 14:59:00.080789 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.080752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd9sc\" (UniqueName: \"kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.080789 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.080792    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.081042 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.080900    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.181752 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.181714    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.181934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.181772    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd9sc\" (UniqueName: \"kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.181934 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.181792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.182492 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.182473    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.184161 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.184145    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.191459 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.191435    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd9sc\" (UniqueName: \"kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc\") pod \"maas-api-e2e-shared-b-06dd21-659d5df485-kw47k\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") " pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.323684 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.323659    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:00.457581 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.457551    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 14:59:00.459291 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:59:00.459267    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8718c7_1d3f_4775_ae24_57a6d6c25fb0.slice/crio-02aac1599b2f98631bb4b09e012fd0c932ccae4c52870bfe9c1ad0fff2f341d2 WatchSource:0}: Error finding container 02aac1599b2f98631bb4b09e012fd0c932ccae4c52870bfe9c1ad0fff2f341d2: Status 404 returned error can't find the container with id 02aac1599b2f98631bb4b09e012fd0c932ccae4c52870bfe9c1ad0fff2f341d2
Jul 06 14:59:00.461105 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.461083    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:59:00.468986 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:00.468962    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" event={"ID":"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0","Type":"ContainerStarted","Data":"02aac1599b2f98631bb4b09e012fd0c932ccae4c52870bfe9c1ad0fff2f341d2"}
Jul 06 14:59:02.481801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:02.481765    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" event={"ID":"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0","Type":"ContainerStarted","Data":"671d48a1dd59f77c3e4873885a1d14bf861e6586157fb5048a7018ad0f583946"}
Jul 06 14:59:02.482191 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:02.481970    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:02.501216 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:02.501168    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" podStartSLOduration=1.909888525 podStartE2EDuration="3.501154039s" totalImagesPullingTime="1.591265514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:59:02.500246428 +0000 UTC m=+2092.709054841" watchObservedRunningTime="2026-07-06 14:59:02.501154039 +0000 UTC m=+2092.709962424"
Jul 06 14:59:08.492553 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:08.492519    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 14:59:10.385682 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:10.385649    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:59:10.388891 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:10.388867    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 14:59:12.613801 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:12.613768    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:59:19.606611 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.606577    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 14:59:19.609807 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.609791    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.614278 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.614251    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"kube-root-ca.crt\""
Jul 06 14:59:19.614426 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.614303    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"openshift-service-ca.crt\""
Jul 06 14:59:19.614426 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.614313    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"default-dockercfg-v85jh\""
Jul 06 14:59:19.614426 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.614314    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"auth-test-model-e63e88-kserve-self-signed-certs\""
Jul 06 14:59:19.620888 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.620861    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 14:59:19.748593 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748561    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.748771 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748604    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psftr\" (UniqueName: \"kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.748771 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748744    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.748849 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748805    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.748849 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748838    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.748914 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.748854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849630 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849601    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849685    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849729    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.849802 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.849763    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psftr\" (UniqueName: \"kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.850122 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.850064    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.850184 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.850143    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.850240 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.850202    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.851957 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.851941    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.852273 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.852256    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.868266 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.868208    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psftr\" (UniqueName: \"kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr\") pod \"auth-test-model-e63e88-kserve-56f7755dbc-hnz6q\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") " pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:19.919540 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:19.919509    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:20.063830 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:20.063801    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 14:59:20.069720 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:59:20.069690    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod409f38e5_f3db_4b7a_bdb2_45a501a73d9e.slice/crio-aae8e9ca7a4ab6061a8ab8285dd76896a5172f797758fffa37ffb952657b1d14 WatchSource:0}: Error finding container aae8e9ca7a4ab6061a8ab8285dd76896a5172f797758fffa37ffb952657b1d14: Status 404 returned error can't find the container with id aae8e9ca7a4ab6061a8ab8285dd76896a5172f797758fffa37ffb952657b1d14
Jul 06 14:59:20.568614 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:20.568586    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" event={"ID":"409f38e5-f3db-4b7a-bdb2-45a501a73d9e","Type":"ContainerStarted","Data":"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"}
Jul 06 14:59:20.568758 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:20.568626    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" event={"ID":"409f38e5-f3db-4b7a-bdb2-45a501a73d9e","Type":"ContainerStarted","Data":"aae8e9ca7a4ab6061a8ab8285dd76896a5172f797758fffa37ffb952657b1d14"}
Jul 06 14:59:20.588966 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:20.588915    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" podStartSLOduration=1.5889001839999999 podStartE2EDuration="1.588900184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:59:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:59:20.587338434 +0000 UTC m=+2110.796146821" watchObservedRunningTime="2026-07-06 14:59:20.588900184 +0000 UTC m=+2110.797708569"
Jul 06 14:59:23.216806 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:23.216770    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 14:59:29.920424 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:29.920374    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:29.920870 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:29.920456    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:29.933113 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:29.933084    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:30.627344 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:30.627309    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 14:59:53.601524 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.601492    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 14:59:53.604788 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.604773    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.608906 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.608876    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"kube-root-ca.crt\""
Jul 06 14:59:53.609075 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.608956    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"auth-test-model-06dd21-kserve-self-signed-certs\""
Jul 06 14:59:53.609075 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.608875    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"default-dockercfg-kxmxh\""
Jul 06 14:59:53.609075 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.608958    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"openshift-service-ca.crt\""
Jul 06 14:59:53.613879 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.613853    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 14:59:53.633777 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633754    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.633898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633785    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.633898 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633820    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.633981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633888    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.633981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633930    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.633981 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.633947    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8djhj\" (UniqueName: \"kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.734849 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.734813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.734872    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.734903    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.734925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8djhj\" (UniqueName: \"kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.734986    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735092 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.735040    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.735329    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735390 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.735333    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.735494 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.735426    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.737276 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.737254    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.737370 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.737332    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.744108 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.744082    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8djhj\" (UniqueName: \"kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj\") pod \"auth-test-model-06dd21-kserve-8648d496bc-hkwfc\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") " pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:53.915265 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:53.915169    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 14:59:54.044232 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:54.044205    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 14:59:54.046406 ip-10-0-140-204 kubenswrapper[2570]: W0706 14:59:54.046374    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2426157d_5722_4372_b4b4_2e67b1f493da.slice/crio-42491baf299eb02f6d79ce5dfadd6be1d9abc59a3021f09c883f7ac94c8e2f07 WatchSource:0}: Error finding container 42491baf299eb02f6d79ce5dfadd6be1d9abc59a3021f09c883f7ac94c8e2f07: Status 404 returned error can't find the container with id 42491baf299eb02f6d79ce5dfadd6be1d9abc59a3021f09c883f7ac94c8e2f07
Jul 06 14:59:54.737485 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:54.737453    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" event={"ID":"2426157d-5722-4372-b4b4-2e67b1f493da","Type":"ContainerStarted","Data":"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"}
Jul 06 14:59:54.737485 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:54.737498    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" event={"ID":"2426157d-5722-4372-b4b4-2e67b1f493da","Type":"ContainerStarted","Data":"42491baf299eb02f6d79ce5dfadd6be1d9abc59a3021f09c883f7ac94c8e2f07"}
Jul 06 14:59:54.762889 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:54.762837    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" podStartSLOduration=1.762818331 podStartE2EDuration="1.762818331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:59:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:59:54.756390348 +0000 UTC m=+2144.965198749" watchObservedRunningTime="2026-07-06 14:59:54.762818331 +0000 UTC m=+2144.971626717"
Jul 06 14:59:56.614754 ip-10-0-140-204 kubenswrapper[2570]: I0706 14:59:56.614721    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:00:03.915712 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:03.915618    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:00:03.915712 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:03.915685    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:00:03.927876 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:03.927852    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:00:04.803956 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:04.803930    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:00:13.127346 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:13.127311    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:00:28.351359 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:28.351321    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:00:45.952941 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:45.952907    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:00:58.828102 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:00:58.828051    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:01:16.925518 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:16.925463    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:01:32.924485 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:32.924403    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:01:50.716798 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:50.716761    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:01:53.399726 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:53.399687    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 15:01:54.508931 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:54.508891    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 15:01:58.697308 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.697275    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:01:58.700781 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.700757    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.703722 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.703699    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"sub-test-model-e63e88-kserve-self-signed-certs\""
Jul 06 15:01:58.718217 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.718186    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:01:58.802681 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802640    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.802874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802692    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.802874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802733    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.802874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802754    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.802874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802774    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5btss\" (UniqueName: \"kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.802874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.802837    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904047 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904008    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904047 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904331 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904087    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904331 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904108    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5btss\" (UniqueName: \"kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904331 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904331 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904177    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904571 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904502    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904571 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904500    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.904693 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.904592    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.906335 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.906316    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.906666 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.906649    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:58.913193 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:58.913171    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5btss\" (UniqueName: \"kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss\") pod \"sub-test-model-e63e88-kserve-7cd8644569-cnd45\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") " pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:59.018634 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:59.018536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:01:59.162299 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:59.162274    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:01:59.163906 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:01:59.163880    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7725a63e_9f8d_4b62_ba15_b8d7ce93204c.slice/crio-24673f8a31463aa81ffe9ea770a46776c23631d0395db0c749970c1de58c64af WatchSource:0}: Error finding container 24673f8a31463aa81ffe9ea770a46776c23631d0395db0c749970c1de58c64af: Status 404 returned error can't find the container with id 24673f8a31463aa81ffe9ea770a46776c23631d0395db0c749970c1de58c64af
Jul 06 15:01:59.379551 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:59.379514    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" event={"ID":"7725a63e-9f8d-4b62-ba15-b8d7ce93204c","Type":"ContainerStarted","Data":"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"}
Jul 06 15:01:59.379551 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:59.379556    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" event={"ID":"7725a63e-9f8d-4b62-ba15-b8d7ce93204c","Type":"ContainerStarted","Data":"24673f8a31463aa81ffe9ea770a46776c23631d0395db0c749970c1de58c64af"}
Jul 06 15:01:59.397608 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:01:59.397556    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" podStartSLOduration=1.3975370599999999 podStartE2EDuration="1.39753706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:01:59.397478686 +0000 UTC m=+2269.606287073" watchObservedRunningTime="2026-07-06 15:01:59.39753706 +0000 UTC m=+2269.606345445"
Jul 06 15:02:06.627372 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:06.627335    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:02:08.420132 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.420083    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" podUID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" containerName="main" containerID="cri-o://d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1" gracePeriod=45
Jul 06 15:02:08.674555 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.674494    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 15:02:08.791339 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791298    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791365    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791384    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791431    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791449    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psftr\" (UniqueName: \"kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791508    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache\") pod \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\" (UID: \"409f38e5-f3db-4b7a-bdb2-45a501a73d9e\") "
Jul 06 15:02:08.791804 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791617    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home" (OuterVolumeSpecName: "home") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:08.791804 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791671    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:08.791804 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791744    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:08.791919 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.791866    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:08.793747 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.793725    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm" (OuterVolumeSpecName: "dshm") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:08.793828 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.793745    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr" (OuterVolumeSpecName: "kube-api-access-psftr") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "kube-api-access-psftr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:02:08.793828 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.793815    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "409f38e5-f3db-4b7a-bdb2-45a501a73d9e" (UID: "409f38e5-f3db-4b7a-bdb2-45a501a73d9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:02:08.892456 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.892417    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:08.892456 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.892449    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psftr\" (UniqueName: \"kubernetes.io/projected/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-kube-api-access-psftr\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:08.892456 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.892459    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:08.892456 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.892470    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:08.892724 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:08.892478    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f38e5-f3db-4b7a-bdb2-45a501a73d9e-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:09.019675 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.019601    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:02:09.019675 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.019642    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:02:09.032542 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.032515    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:02:09.435608 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.435575    2570 generic.go:356] "Generic (PLEG): container finished" podID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" containerID="d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1" exitCode=0
Jul 06 15:02:09.436140 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.435617    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"
Jul 06 15:02:09.436140 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.435646    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" event={"ID":"409f38e5-f3db-4b7a-bdb2-45a501a73d9e","Type":"ContainerDied","Data":"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"}
Jul 06 15:02:09.436140 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.435677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q" event={"ID":"409f38e5-f3db-4b7a-bdb2-45a501a73d9e","Type":"ContainerDied","Data":"aae8e9ca7a4ab6061a8ab8285dd76896a5172f797758fffa37ffb952657b1d14"}
Jul 06 15:02:09.436140 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.435693    2570 scope.go:117] "RemoveContainer" containerID="d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"
Jul 06 15:02:09.450851 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.450822    2570 scope.go:117] "RemoveContainer" containerID="d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"
Jul 06 15:02:09.451065 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.451050    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:02:09.451223 ip-10-0-140-204 kubenswrapper[2570]: E0706 15:02:09.451198    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1\": container with ID starting with d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1 not found: ID does not exist" containerID="d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"
Jul 06 15:02:09.451266 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.451233    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1"} err="failed to get container status \"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1\": rpc error: code = NotFound desc = could not find container \"d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1\": container with ID starting with d00832ff94fe522a70b28370ff22d3d4471f0e8ecd2ecdac60043c4c36c2e4a1 not found: ID does not exist"
Jul 06 15:02:09.467058 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.467023    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 15:02:09.470531 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.470509    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/auth-test-model-e63e88-kserve-56f7755dbc-hnz6q"]
Jul 06 15:02:09.529761 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.529723    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" podUID="2426157d-5722-4372-b4b4-2e67b1f493da" containerName="main" containerID="cri-o://86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f" gracePeriod=45
Jul 06 15:02:09.779529 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.779500    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:02:09.902313 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902277    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902507 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902330    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8djhj\" (UniqueName: \"kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902507 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902368    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902630 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902528    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902630 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902563    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902630 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902610    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir\") pod \"2426157d-5722-4372-b4b4-2e67b1f493da\" (UID: \"2426157d-5722-4372-b4b4-2e67b1f493da\") "
Jul 06 15:02:09.902833 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902655    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home" (OuterVolumeSpecName: "home") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:09.902908 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902884    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:09.902908 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902871    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache" (OuterVolumeSpecName: "model-cache") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:09.903351 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.902927    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:09.904722 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.904699    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:02:09.904722 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.904713    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm" (OuterVolumeSpecName: "dshm") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:02:09.904865 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:09.904762    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj" (OuterVolumeSpecName: "kube-api-access-8djhj") pod "2426157d-5722-4372-b4b4-2e67b1f493da" (UID: "2426157d-5722-4372-b4b4-2e67b1f493da"). InnerVolumeSpecName "kube-api-access-8djhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:02:10.004166 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.004065    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:10.004166 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.004106    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:10.004166 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.004117    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2426157d-5722-4372-b4b4-2e67b1f493da-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:10.004166 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.004124    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2426157d-5722-4372-b4b4-2e67b1f493da-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:10.004166 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.004132    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8djhj\" (UniqueName: \"kubernetes.io/projected/2426157d-5722-4372-b4b4-2e67b1f493da-kube-api-access-8djhj\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:02:10.292533 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.292448    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" path="/var/lib/kubelet/pods/409f38e5-f3db-4b7a-bdb2-45a501a73d9e/volumes"
Jul 06 15:02:10.443080 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.443046    2570 generic.go:356] "Generic (PLEG): container finished" podID="2426157d-5722-4372-b4b4-2e67b1f493da" containerID="86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f" exitCode=0
Jul 06 15:02:10.443542 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.443090    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"
Jul 06 15:02:10.443542 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.443123    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" event={"ID":"2426157d-5722-4372-b4b4-2e67b1f493da","Type":"ContainerDied","Data":"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"}
Jul 06 15:02:10.443542 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.443163    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc" event={"ID":"2426157d-5722-4372-b4b4-2e67b1f493da","Type":"ContainerDied","Data":"42491baf299eb02f6d79ce5dfadd6be1d9abc59a3021f09c883f7ac94c8e2f07"}
Jul 06 15:02:10.443542 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.443194    2570 scope.go:117] "RemoveContainer" containerID="86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"
Jul 06 15:02:10.451761 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.451737    2570 scope.go:117] "RemoveContainer" containerID="86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"
Jul 06 15:02:10.452034 ip-10-0-140-204 kubenswrapper[2570]: E0706 15:02:10.452017    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f\": container with ID starting with 86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f not found: ID does not exist" containerID="86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"
Jul 06 15:02:10.452121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.452042    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f"} err="failed to get container status \"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f\": rpc error: code = NotFound desc = could not find container \"86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f\": container with ID starting with 86ee5c3fc24c44d893801c3943b34e5ebab4250617d75beb1238ef76f6bfc37f not found: ID does not exist"
Jul 06 15:02:10.462263 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.462238    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 15:02:10.466431 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:10.466400    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/auth-test-model-06dd21-kserve-8648d496bc-hkwfc"]
Jul 06 15:02:12.292655 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:12.292615    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2426157d-5722-4372-b4b4-2e67b1f493da" path="/var/lib/kubelet/pods/2426157d-5722-4372-b4b4-2e67b1f493da/volumes"
Jul 06 15:02:22.805807 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.805773    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806117    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2426157d-5722-4372-b4b4-2e67b1f493da" containerName="main"
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806128    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2426157d-5722-4372-b4b4-2e67b1f493da" containerName="main"
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806139    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" containerName="main"
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806145    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" containerName="main"
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806205    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2426157d-5722-4372-b4b4-2e67b1f493da" containerName="main"
Jul 06 15:02:22.806297 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.806215    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="409f38e5-f3db-4b7a-bdb2-45a501a73d9e" containerName="main"
Jul 06 15:02:22.809110 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.809093    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.814043 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.814016    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"default-dockercfg-kxmxh\""
Jul 06 15:02:22.814043 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.814021    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"kube-root-ca.crt\""
Jul 06 15:02:22.814256 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.814063    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"openshift-service-ca.crt\""
Jul 06 15:02:22.814256 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.814086    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"sub-test-model-06dd21-kserve-self-signed-certs\""
Jul 06 15:02:22.820158 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.820129    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:02:22.920836 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.920795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.921063 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.920842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.921063 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.920863    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.921063 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.920935    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.921063 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.920986    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:22.921210 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:22.921073    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvvpl\" (UniqueName: \"kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022505    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022544 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022554    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvvpl\" (UniqueName: \"kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022733    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.022822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.022755    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.023079 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.023053    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.023123 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.023100    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.023170 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.023125    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.024794 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.024773    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.025073 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.025053    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.032279 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.032253    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvvpl\" (UniqueName: \"kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl\") pod \"sub-test-model-06dd21-kserve-76bc9cccfc-8frqz\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") " pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.119055 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.118979    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:23.268563 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.268534    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:02:23.271139 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:02:23.271113    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e4d2f2c_8dfc_4ec8_ba0f_0acf585ea916.slice/crio-84bb9cbeeaa5423b0353af0f64d12849ca7574229efb44c95ac9cb5b5c7540b2 WatchSource:0}: Error finding container 84bb9cbeeaa5423b0353af0f64d12849ca7574229efb44c95ac9cb5b5c7540b2: Status 404 returned error can't find the container with id 84bb9cbeeaa5423b0353af0f64d12849ca7574229efb44c95ac9cb5b5c7540b2
Jul 06 15:02:23.511847 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.511750    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" event={"ID":"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916","Type":"ContainerStarted","Data":"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"}
Jul 06 15:02:23.511847 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.511797    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" event={"ID":"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916","Type":"ContainerStarted","Data":"84bb9cbeeaa5423b0353af0f64d12849ca7574229efb44c95ac9cb5b5c7540b2"}
Jul 06 15:02:23.532156 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:23.532103    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" podStartSLOduration=1.532088619 podStartE2EDuration="1.532088619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:02:23.530973733 +0000 UTC m=+2293.739782119" watchObservedRunningTime="2026-07-06 15:02:23.532088619 +0000 UTC m=+2293.740897002"
Jul 06 15:02:27.413397 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:27.413347    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:02:33.120052 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:33.119981    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:33.120534 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:33.120079    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:33.132854 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:33.132825    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:33.576180 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:33.576154    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:02:41.949536 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:41.949497    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:02:58.845084 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:58.845039    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:02:59.718332 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:02:59.718300    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:03:01.730188 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:01.730153    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:03:09.187843 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.187806    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:03:09.191772 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.191754    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.194517 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.194486    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e63e88\"/\"rate-test-model-e63e88-kserve-self-signed-certs\""
Jul 06 15:03:09.200025 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.199910    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:03:09.337273 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.337453 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337292    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.337453 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.337453 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.337453 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.337453 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.337450    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.438946 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.438848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.438946 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.438925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.438960    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439020    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439047    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439073    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439585 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439540    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439585 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439561    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.439782 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.439708    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.441448 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.441422    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.442154 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.442134    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.450855 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.450831    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl\") pod \"rate-test-model-e63e88-kserve-74b87bcd99-zrc69\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") " pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.503137 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.503100    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:09.645409 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.645373    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:03:09.648520 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:03:09.648495    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4feb675b_7a6d_4385_a98e_a5a0a88a5bcb.slice/crio-d844e92386cdf1d5064d6747bf897ba110f56540af5eef1faa515d542b4a5c5c WatchSource:0}: Error finding container d844e92386cdf1d5064d6747bf897ba110f56540af5eef1faa515d542b4a5c5c: Status 404 returned error can't find the container with id d844e92386cdf1d5064d6747bf897ba110f56540af5eef1faa515d542b4a5c5c
Jul 06 15:03:09.742174 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.742141    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" event={"ID":"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb","Type":"ContainerStarted","Data":"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"}
Jul 06 15:03:09.742382 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.742193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" event={"ID":"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb","Type":"ContainerStarted","Data":"d844e92386cdf1d5064d6747bf897ba110f56540af5eef1faa515d542b4a5c5c"}
Jul 06 15:03:09.762812 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:09.762762    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" podStartSLOduration=0.762746651 podStartE2EDuration="762.746651ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:03:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:03:09.760642006 +0000 UTC m=+2339.969450398" watchObservedRunningTime="2026-07-06 15:03:09.762746651 +0000 UTC m=+2339.971555036"
Jul 06 15:03:13.865110 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:13.865065    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" podUID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" containerName="main" containerID="cri-o://34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab" gracePeriod=45
Jul 06 15:03:14.118882 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.118814    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:03:14.288434 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288403    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5btss\" (UniqueName: \"kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288652 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288465    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288652 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288489    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288652 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288510    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288652 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288640    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288700    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home\") pod \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\" (UID: \"7725a63e-9f8d-4b62-ba15-b8d7ce93204c\") "
Jul 06 15:03:14.288874 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288823    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache" (OuterVolumeSpecName: "model-cache") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:14.289028 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.288900    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:14.289097 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.289049    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home" (OuterVolumeSpecName: "home") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:14.289153 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.289109    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.289153 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.289131    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.290657 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.290632    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:03:14.290794 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.290703    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm" (OuterVolumeSpecName: "dshm") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:14.290794 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.290725    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss" (OuterVolumeSpecName: "kube-api-access-5btss") pod "7725a63e-9f8d-4b62-ba15-b8d7ce93204c" (UID: "7725a63e-9f8d-4b62-ba15-b8d7ce93204c"). InnerVolumeSpecName "kube-api-access-5btss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:03:14.390347 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.390260    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.390347 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.390285    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.390347 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.390296    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5btss\" (UniqueName: \"kubernetes.io/projected/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-kube-api-access-5btss\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.390347 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.390306    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7725a63e-9f8d-4b62-ba15-b8d7ce93204c-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:14.738916 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.738816    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" podUID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" containerName="main" containerID="cri-o://05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376" gracePeriod=45
Jul 06 15:03:14.776821 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.776783    2570 generic.go:356] "Generic (PLEG): container finished" podID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" containerID="34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab" exitCode=0
Jul 06 15:03:14.776821 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.776826    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"
Jul 06 15:03:14.777052 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.776872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" event={"ID":"7725a63e-9f8d-4b62-ba15-b8d7ce93204c","Type":"ContainerDied","Data":"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"}
Jul 06 15:03:14.777052 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.776910    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45" event={"ID":"7725a63e-9f8d-4b62-ba15-b8d7ce93204c","Type":"ContainerDied","Data":"24673f8a31463aa81ffe9ea770a46776c23631d0395db0c749970c1de58c64af"}
Jul 06 15:03:14.777052 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.776931    2570 scope.go:117] "RemoveContainer" containerID="34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"
Jul 06 15:03:14.817132 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.817096    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:03:14.826635 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.826609    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/sub-test-model-e63e88-kserve-7cd8644569-cnd45"]
Jul 06 15:03:14.850831 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.850806    2570 scope.go:117] "RemoveContainer" containerID="34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"
Jul 06 15:03:14.851221 ip-10-0-140-204 kubenswrapper[2570]: E0706 15:03:14.851196    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab\": container with ID starting with 34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab not found: ID does not exist" containerID="34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"
Jul 06 15:03:14.851339 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.851228    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab"} err="failed to get container status \"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab\": rpc error: code = NotFound desc = could not find container \"34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab\": container with ID starting with 34fe6aef644bf339323a2a7d8d8bcfd641fd68f7d46709e8d5c502afa47fcdab not found: ID does not exist"
Jul 06 15:03:14.988869 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:14.988840    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:03:15.095973 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.095936    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096199 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096044    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096199 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096108    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvvpl\" (UniqueName: \"kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096199 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096138    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096199 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096169    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096420 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096251    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm\") pod \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\" (UID: \"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916\") "
Jul 06 15:03:15.096420 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096254    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:15.096420 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096362    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home" (OuterVolumeSpecName: "home") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:15.096420 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096396    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:15.096596 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096523    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.096596 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096540    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.096596 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.096552    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.098292 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.098266    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm" (OuterVolumeSpecName: "dshm") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:03:15.098408 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.098357    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:03:15.098408 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.098402    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl" (OuterVolumeSpecName: "kube-api-access-lvvpl") pod "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" (UID: "7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916"). InnerVolumeSpecName "kube-api-access-lvvpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:03:15.197536 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.197505    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvvpl\" (UniqueName: \"kubernetes.io/projected/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-kube-api-access-lvvpl\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.197536 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.197533    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.197536 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.197544    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:03:15.783753 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.783716    2570 generic.go:356] "Generic (PLEG): container finished" podID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" containerID="05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376" exitCode=0
Jul 06 15:03:15.783944 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.783761    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"
Jul 06 15:03:15.783944 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.783776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" event={"ID":"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916","Type":"ContainerDied","Data":"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"}
Jul 06 15:03:15.783944 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.783816    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz" event={"ID":"7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916","Type":"ContainerDied","Data":"84bb9cbeeaa5423b0353af0f64d12849ca7574229efb44c95ac9cb5b5c7540b2"}
Jul 06 15:03:15.783944 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.783840    2570 scope.go:117] "RemoveContainer" containerID="05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"
Jul 06 15:03:15.793335 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.793315    2570 scope.go:117] "RemoveContainer" containerID="05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"
Jul 06 15:03:15.793568 ip-10-0-140-204 kubenswrapper[2570]: E0706 15:03:15.793550    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376\": container with ID starting with 05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376 not found: ID does not exist" containerID="05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"
Jul 06 15:03:15.793605 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.793576    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376"} err="failed to get container status \"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376\": rpc error: code = NotFound desc = could not find container \"05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376\": container with ID starting with 05f61779b132089dc5869fb2bd305af54c7ecfb4f9317f4d30b1a87b9f29d376 not found: ID does not exist"
Jul 06 15:03:15.808640 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.808609    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:03:15.812606 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:15.812582    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/sub-test-model-06dd21-kserve-76bc9cccfc-8frqz"]
Jul 06 15:03:16.293491 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:16.293457    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" path="/var/lib/kubelet/pods/7725a63e-9f8d-4b62-ba15-b8d7ce93204c/volumes"
Jul 06 15:03:16.293877 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:16.293842    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" path="/var/lib/kubelet/pods/7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916/volumes"
Jul 06 15:03:19.503641 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:19.503603    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:19.504093 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:19.503651    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:19.516113 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:19.516081    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:19.822272 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:19.822249    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:03:22.919748 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:22.919709    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:03:24.497882 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.497846    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:03:24.498371 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498329    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" containerName="main"
Jul 06 15:03:24.498371 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498348    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" containerName="main"
Jul 06 15:03:24.498497 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498378    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" containerName="main"
Jul 06 15:03:24.498497 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498387    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" containerName="main"
Jul 06 15:03:24.498497 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498471    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4d2f2c-8dfc-4ec8-ba0f-0acf585ea916" containerName="main"
Jul 06 15:03:24.498497 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.498485    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7725a63e-9f8d-4b62-ba15-b8d7ce93204c" containerName="main"
Jul 06 15:03:24.501644 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.501624    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.504459 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.504436    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"kube-root-ca.crt\""
Jul 06 15:03:24.504628 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.504609    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"openshift-service-ca.crt\""
Jul 06 15:03:24.505939 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.505918    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"default-dockercfg-kxmxh\""
Jul 06 15:03:24.506058 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.505977    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-06dd21\"/\"rate-test-model-06dd21-kserve-self-signed-certs\""
Jul 06 15:03:24.511814 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.511792    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:03:24.572221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572190    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.572396 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572226    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.572396 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.572396 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572290    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzbpp\" (UniqueName: \"kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.572396 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572322    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.572396 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.572368    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673597 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673554    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673597 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673597    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673856 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673615    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673856 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673644    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzbpp\" (UniqueName: \"kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673856 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.673856 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.673701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.674117 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.674058    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.674117 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.674088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.674189 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.674149    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.676059 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.676038    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.676279 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.676265    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.682985 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.682964    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzbpp\" (UniqueName: \"kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp\") pod \"rate-test-model-06dd21-kserve-b8cf9b659-z62c5\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") " pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.811117 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.811034    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:24.951276 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:24.951249    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:03:24.954025 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:03:24.953985    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2934a4fa_0fb6_426a_b515_e1ff9bc8de24.slice/crio-bc5d65f8337bfd4cae97c9eb9fa5e71edaafffa4c64e06bbd398e781d75a73a1 WatchSource:0}: Error finding container bc5d65f8337bfd4cae97c9eb9fa5e71edaafffa4c64e06bbd398e781d75a73a1: Status 404 returned error can't find the container with id bc5d65f8337bfd4cae97c9eb9fa5e71edaafffa4c64e06bbd398e781d75a73a1
Jul 06 15:03:25.844797 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:25.844758    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" event={"ID":"2934a4fa-0fb6-426a-b515-e1ff9bc8de24","Type":"ContainerStarted","Data":"2c2114c2f55bd1162692921dc120ced90b79e9d461182011fef27e09cf1cf716"}
Jul 06 15:03:25.845247 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:25.844806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" event={"ID":"2934a4fa-0fb6-426a-b515-e1ff9bc8de24","Type":"ContainerStarted","Data":"bc5d65f8337bfd4cae97c9eb9fa5e71edaafffa4c64e06bbd398e781d75a73a1"}
Jul 06 15:03:25.865451 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:25.865345    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" podStartSLOduration=1.865328839 podStartE2EDuration="1.865328839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:03:25.8644364 +0000 UTC m=+2356.073244787" watchObservedRunningTime="2026-07-06 15:03:25.865328839 +0000 UTC m=+2356.074137224"
Jul 06 15:03:34.811731 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:34.811692    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:34.812390 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:34.811745    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:34.824687 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:34.824663    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:34.900347 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:34.900318    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:03:35.710649 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:35.710612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:03:59.248443 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:03:59.248410    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:04:10.418059 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:10.418030    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 15:04:10.423161 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:10.423140    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 15:04:12.014244 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:12.014200    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:04:19.247121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:19.247087    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:04:20.027967 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:20.027936    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:04:25.513568 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:25.513532    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:04:34.266624 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.266514    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" podUID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" containerName="main" containerID="cri-o://82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127" gracePeriod=45
Jul 06 15:04:34.511715 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.511691    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:04:34.587822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587787    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.587822 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587826    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.588121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587844    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.588121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587910    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.588121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587931    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.588121 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.587961    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl\") pod \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\" (UID: \"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb\") "
Jul 06 15:04:34.588325 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.588177    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache" (OuterVolumeSpecName: "model-cache") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:34.588325 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.588230    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:34.588325 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.588249    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home" (OuterVolumeSpecName: "home") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:34.590047 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.590022    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:04:34.590161 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.590027    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm" (OuterVolumeSpecName: "dshm") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:34.590161 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.590068    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl" (OuterVolumeSpecName: "kube-api-access-vmdjl") pod "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" (UID: "4feb675b-7a6d-4385-a98e-a5a0a88a5bcb"). InnerVolumeSpecName "kube-api-access-vmdjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:04:34.688694 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688659    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:34.688694 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688690    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:34.688694 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688699    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-kube-api-access-vmdjl\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:34.688926 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688708    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:34.688926 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688717    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:34.688926 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:34.688725    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.047896 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.047789    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" podUID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" containerName="main" containerID="cri-o://2c2114c2f55bd1162692921dc120ced90b79e9d461182011fef27e09cf1cf716" gracePeriod=45
Jul 06 15:04:35.192852 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.192691    2570 generic.go:356] "Generic (PLEG): container finished" podID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" containerID="82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127" exitCode=0
Jul 06 15:04:35.192852 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.192736    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"
Jul 06 15:04:35.192852 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.192797    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" event={"ID":"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb","Type":"ContainerDied","Data":"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"}
Jul 06 15:04:35.192852 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.192830    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69" event={"ID":"4feb675b-7a6d-4385-a98e-a5a0a88a5bcb","Type":"ContainerDied","Data":"d844e92386cdf1d5064d6747bf897ba110f56540af5eef1faa515d542b4a5c5c"}
Jul 06 15:04:35.192852 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.192851    2570 scope.go:117] "RemoveContainer" containerID="82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"
Jul 06 15:04:35.195019 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.194970    2570 generic.go:356] "Generic (PLEG): container finished" podID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" containerID="2c2114c2f55bd1162692921dc120ced90b79e9d461182011fef27e09cf1cf716" exitCode=0
Jul 06 15:04:35.195127 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.195024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" event={"ID":"2934a4fa-0fb6-426a-b515-e1ff9bc8de24","Type":"ContainerDied","Data":"2c2114c2f55bd1162692921dc120ced90b79e9d461182011fef27e09cf1cf716"}
Jul 06 15:04:35.202245 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.202225    2570 scope.go:117] "RemoveContainer" containerID="82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"
Jul 06 15:04:35.202509 ip-10-0-140-204 kubenswrapper[2570]: E0706 15:04:35.202489    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127\": container with ID starting with 82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127 not found: ID does not exist" containerID="82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"
Jul 06 15:04:35.202554 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.202517    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127"} err="failed to get container status \"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127\": rpc error: code = NotFound desc = could not find container \"82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127\": container with ID starting with 82219ea2583a36a374f4a887e57699ed1c7c50b7562b15fe7478f863e8325127 not found: ID does not exist"
Jul 06 15:04:35.224741 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.224712    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:04:35.239516 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.239486    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e63e88/rate-test-model-e63e88-kserve-74b87bcd99-zrc69"]
Jul 06 15:04:35.295586 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.295563    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:04:35.394480 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394444    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394682 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394496    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394682 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394530    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzbpp\" (UniqueName: \"kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394682 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394597    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394682 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394637    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394920 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394686    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home\") pod \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\" (UID: \"2934a4fa-0fb6-426a-b515-e1ff9bc8de24\") "
Jul 06 15:04:35.394983 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.394940    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:35.395201 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.395131    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home" (OuterVolumeSpecName: "home") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:35.395344 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.395319    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache" (OuterVolumeSpecName: "model-cache") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:35.395449 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.395417    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tmp-dir\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.395621 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.395608    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-model-cache\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.395713 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.395703    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-home\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.397425 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.397398    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp" (OuterVolumeSpecName: "kube-api-access-nzbpp") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "kube-api-access-nzbpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:04:35.397636 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.397609    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:04:35.397636 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.397617    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm" (OuterVolumeSpecName: "dshm") pod "2934a4fa-0fb6-426a-b515-e1ff9bc8de24" (UID: "2934a4fa-0fb6-426a-b515-e1ff9bc8de24"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:04:35.496592 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.496554    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-dshm\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.496592 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.496583    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-tls-certs\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:35.496592 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:35.496596    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzbpp\" (UniqueName: \"kubernetes.io/projected/2934a4fa-0fb6-426a-b515-e1ff9bc8de24-kube-api-access-nzbpp\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:04:36.203763 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.203731    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"
Jul 06 15:04:36.203941 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.203725    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5" event={"ID":"2934a4fa-0fb6-426a-b515-e1ff9bc8de24","Type":"ContainerDied","Data":"bc5d65f8337bfd4cae97c9eb9fa5e71edaafffa4c64e06bbd398e781d75a73a1"}
Jul 06 15:04:36.203941 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.203861    2570 scope.go:117] "RemoveContainer" containerID="2c2114c2f55bd1162692921dc120ced90b79e9d461182011fef27e09cf1cf716"
Jul 06 15:04:36.229502 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.229474    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:04:36.232019 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.231977    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-06dd21/rate-test-model-06dd21-kserve-b8cf9b659-z62c5"]
Jul 06 15:04:36.292724 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.292692    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" path="/var/lib/kubelet/pods/2934a4fa-0fb6-426a-b515-e1ff9bc8de24/volumes"
Jul 06 15:04:36.293202 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:04:36.293185    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" path="/var/lib/kubelet/pods/4feb675b-7a6d-4385-a98e-a5a0a88a5bcb/volumes"
Jul 06 15:05:04.992039 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:04.992002    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 15:05:04.992619 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:04.992242    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" podUID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" containerName="maas-api" containerID="cri-o://671d48a1dd59f77c3e4873885a1d14bf861e6586157fb5048a7018ad0f583946" gracePeriod=30
Jul 06 15:05:05.350561 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.350529    2570 generic.go:356] "Generic (PLEG): container finished" podID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" containerID="671d48a1dd59f77c3e4873885a1d14bf861e6586157fb5048a7018ad0f583946" exitCode=0
Jul 06 15:05:05.350742 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.350609    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" event={"ID":"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0","Type":"ContainerDied","Data":"671d48a1dd59f77c3e4873885a1d14bf861e6586157fb5048a7018ad0f583946"}
Jul 06 15:05:05.438884 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.438861    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 15:05:05.555255 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.555163    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd9sc\" (UniqueName: \"kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc\") pod \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") "
Jul 06 15:05:05.555255 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.555214    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca\") pod \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") "
Jul 06 15:05:05.555255 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.555240    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls\") pod \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\" (UID: \"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0\") "
Jul 06 15:05:05.555654 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.555628    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" (UID: "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:05:05.557469 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.557450    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" (UID: "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:05:05.557557 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.557537    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc" (OuterVolumeSpecName: "kube-api-access-zd9sc") pod "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" (UID: "1f8718c7-1d3f-4775-ae24-57a6d6c25fb0"). InnerVolumeSpecName "kube-api-access-zd9sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:05:05.656412 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.656371    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd9sc\" (UniqueName: \"kubernetes.io/projected/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-kube-api-access-zd9sc\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:05:05.656412 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.656408    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-openshift-service-ca\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:05:05.656412 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:05.656421    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0-maas-api-tls\") on node \"ip-10-0-140-204.ec2.internal\" DevicePath \"\""
Jul 06 15:05:06.360114 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.360088    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"
Jul 06 15:05:06.360527 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.360136    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k" event={"ID":"1f8718c7-1d3f-4775-ae24-57a6d6c25fb0","Type":"ContainerDied","Data":"02aac1599b2f98631bb4b09e012fd0c932ccae4c52870bfe9c1ad0fff2f341d2"}
Jul 06 15:05:06.360527 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.360203    2570 scope.go:117] "RemoveContainer" containerID="671d48a1dd59f77c3e4873885a1d14bf861e6586157fb5048a7018ad0f583946"
Jul 06 15:05:06.379353 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.379323    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 15:05:06.384239 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.384213    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-06dd21-659d5df485-kw47k"]
Jul 06 15:05:06.917448 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:06.917416    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qg5ks"]
Jul 06 15:05:08.292367 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:08.292334    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" path="/var/lib/kubelet/pods/1f8718c7-1d3f-4775-ae24-57a6d6c25fb0/volumes"
Jul 06 15:05:12.246413 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:12.246378    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-545756f545-qhhcq_ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1/authorino/0.log"
Jul 06 15:05:17.764198 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:17.764169    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mkl2l_95cc674f-1537-4e2a-b4b6-ddf361f4b5ff/server/0.log"
Jul 06 15:05:17.887311 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:17.887281    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kb6f4_ae0395e6-8234-4c18-9476-b9929fadc979/manager/0.log"
Jul 06 15:05:18.144262 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:18.144232    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-flqn8_ab11157e-a292-4369-bf5c-be0013bfaef2/copy-manifests/0.log"
Jul 06 15:05:18.158720 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:18.158693    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-flqn8_ab11157e-a292-4369-bf5c-be0013bfaef2/manager/0.log"
Jul 06 15:05:19.332404 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.332379    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/util/0.log"
Jul 06 15:05:19.340369 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.340345    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/pull/0.log"
Jul 06 15:05:19.349209 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.349148    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/extract/0.log"
Jul 06 15:05:19.489443 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.489407    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/pull/0.log"
Jul 06 15:05:19.496539 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.496508    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/extract/0.log"
Jul 06 15:05:19.504048 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.504021    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/util/0.log"
Jul 06 15:05:19.647510 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.647433    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/util/0.log"
Jul 06 15:05:19.656617 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.656593    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/pull/0.log"
Jul 06 15:05:19.667057 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.667032    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/extract/0.log"
Jul 06 15:05:19.794373 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.794344    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/util/0.log"
Jul 06 15:05:19.807169 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.807144    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/pull/0.log"
Jul 06 15:05:19.817379 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.817350    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/extract/0.log"
Jul 06 15:05:19.945953 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:19.945877    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-545756f545-qhhcq_ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1/authorino/0.log"
Jul 06 15:05:20.688274 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:20.688249    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qg5ks_dd8f3bea-92c0-4e31-b0fe-cb89de9b5246/limitador/0.log"
Jul 06 15:05:21.561036 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:21.560985    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d848d496-gfwqq_151c4ae4-f7fe-4bdd-8cd6-3919ded59618/kube-auth-proxy/0.log"
Jul 06 15:05:26.877984 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.877950    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8frwf/must-gather-4ppc9"]
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878309    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" containerName="maas-api"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878322    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" containerName="maas-api"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878331    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" containerName="main"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878336    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" containerName="main"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878349    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" containerName="main"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878355    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" containerName="main"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878409    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8718c7-1d3f-4775-ae24-57a6d6c25fb0" containerName="maas-api"
Jul 06 15:05:26.878414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878420    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4feb675b-7a6d-4385-a98e-a5a0a88a5bcb" containerName="main"
Jul 06 15:05:26.878691 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.878427    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2934a4fa-0fb6-426a-b515-e1ff9bc8de24" containerName="main"
Jul 06 15:05:26.881498 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.881478    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:26.884428 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.884412    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8frwf\"/\"kube-root-ca.crt\""
Jul 06 15:05:26.884527 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.884412    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8frwf\"/\"openshift-service-ca.crt\""
Jul 06 15:05:26.885606 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.885592    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8frwf\"/\"default-dockercfg-dstt7\""
Jul 06 15:05:26.889127 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.889104    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/must-gather-4ppc9"]
Jul 06 15:05:26.913315 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.913244    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6c30bd88-6851-4f1b-a61d-94084d7ece13-must-gather-output\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:26.913516 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:26.913427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcn8\" (UniqueName: \"kubernetes.io/projected/6c30bd88-6851-4f1b-a61d-94084d7ece13-kube-api-access-frcn8\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.014339 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.014298    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcn8\" (UniqueName: \"kubernetes.io/projected/6c30bd88-6851-4f1b-a61d-94084d7ece13-kube-api-access-frcn8\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.014519 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.014361    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6c30bd88-6851-4f1b-a61d-94084d7ece13-must-gather-output\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.014680 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.014664    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6c30bd88-6851-4f1b-a61d-94084d7ece13-must-gather-output\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.023395 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.023361    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcn8\" (UniqueName: \"kubernetes.io/projected/6c30bd88-6851-4f1b-a61d-94084d7ece13-kube-api-access-frcn8\") pod \"must-gather-4ppc9\" (UID: \"6c30bd88-6851-4f1b-a61d-94084d7ece13\") " pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.192143 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.192050    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/must-gather-4ppc9"
Jul 06 15:05:27.336034 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.336005    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/must-gather-4ppc9"]
Jul 06 15:05:27.337838 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:05:27.337804    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c30bd88_6851_4f1b_a61d_94084d7ece13.slice/crio-e2a3d1e97e07a0568ae29b839686efd4244e4a438d91483a3589b7c20dd65ecb WatchSource:0}: Error finding container e2a3d1e97e07a0568ae29b839686efd4244e4a438d91483a3589b7c20dd65ecb: Status 404 returned error can't find the container with id e2a3d1e97e07a0568ae29b839686efd4244e4a438d91483a3589b7c20dd65ecb
Jul 06 15:05:27.339966 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.339949    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 15:05:27.466230 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:27.466140    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/must-gather-4ppc9" event={"ID":"6c30bd88-6851-4f1b-a61d-94084d7ece13","Type":"ContainerStarted","Data":"e2a3d1e97e07a0568ae29b839686efd4244e4a438d91483a3589b7c20dd65ecb"}
Jul 06 15:05:28.475734 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:28.475211    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/must-gather-4ppc9" event={"ID":"6c30bd88-6851-4f1b-a61d-94084d7ece13","Type":"ContainerStarted","Data":"1f5164db77c6d5b7a5aba72c45c868743693d2e800e7b57dbe24d75431b4aff0"}
Jul 06 15:05:28.475734 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:28.475264    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/must-gather-4ppc9" event={"ID":"6c30bd88-6851-4f1b-a61d-94084d7ece13","Type":"ContainerStarted","Data":"4f2c7c22ec0742d047107ba4c13ee919f5d10ac664bd547914c2b3151dff3f79"}
Jul 06 15:05:29.853254 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:29.853225    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wlxc7_c3a36fc8-b9bd-47d1-963e-46d65a8867dc/global-pull-secret-syncer/0.log"
Jul 06 15:05:29.942064 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:29.942030    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d8xgh_523eaa34-2dd5-4ec3-b9c1-4207ee722ef7/konnectivity-agent/0.log"
Jul 06 15:05:30.068365 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:30.068325    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-204.ec2.internal_7ee29cf4a0b9a74c02a59c261bf9c8fd/haproxy/0.log"
Jul 06 15:05:34.960850 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:34.960820    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/extract/0.log"
Jul 06 15:05:34.987368 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:34.987331    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/util/0.log"
Jul 06 15:05:35.017135 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.017106    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s2l56_ec98fb49-f08c-4a2d-b985-54f95f6110ad/pull/0.log"
Jul 06 15:05:35.056085 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.056048    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/extract/0.log"
Jul 06 15:05:35.082426 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.082393    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/util/0.log"
Jul 06 15:05:35.109616 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.109585    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7q5w_bb488e0f-4ed2-4b35-9fae-40d3e88f3d78/pull/0.log"
Jul 06 15:05:35.142929 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.142890    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/extract/0.log"
Jul 06 15:05:35.170221 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.170186    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/util/0.log"
Jul 06 15:05:35.199867 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.199838    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vxflz_7d53eb7a-257f-4cfd-af88-6165d8616467/pull/0.log"
Jul 06 15:05:35.232403 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.232314    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/extract/0.log"
Jul 06 15:05:35.259043 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.259011    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/util/0.log"
Jul 06 15:05:35.290777 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.290713    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtbw7_e99c0895-6050-4811-bf21-509bec6bdfdc/pull/0.log"
Jul 06 15:05:35.699847 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:35.699810    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-545756f545-qhhcq_ceea96f3-4ef8-4fcb-a7f5-00f36a5f5ed1/authorino/0.log"
Jul 06 15:05:36.084150 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:36.084109    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qg5ks_dd8f3bea-92c0-4e31-b0fe-cb89de9b5246/limitador/0.log"
Jul 06 15:05:37.991841 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:37.991807    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wpkpg_0d65dcd3-852a-4605-86f9-fc9307cffad4/cluster-monitoring-operator/0.log"
Jul 06 15:05:38.310767 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:38.310690    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fpmk_d39a271f-1e65-47b0-8b2c-6c558ab5ecd1/node-exporter/0.log"
Jul 06 15:05:38.335400 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:38.335367    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fpmk_d39a271f-1e65-47b0-8b2c-6c558ab5ecd1/kube-rbac-proxy/0.log"
Jul 06 15:05:38.361042 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:38.361011    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fpmk_d39a271f-1e65-47b0-8b2c-6c558ab5ecd1/init-textfile/0.log"
Jul 06 15:05:39.322014 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.321309    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8frwf/must-gather-4ppc9" podStartSLOduration=12.621684947 podStartE2EDuration="13.321285407s" totalImagesPullingTime="699.60046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:05:28.494054258 +0000 UTC m=+2478.702862645" watchObservedRunningTime="2026-07-06 15:05:39.321285407 +0000 UTC m=+2489.530093794"
Jul 06 15:05:39.324444 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.323771    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"]
Jul 06 15:05:39.329908 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.329883    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.339012 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.338328    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"]
Jul 06 15:05:39.429468 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.429435    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-podres\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.429827 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.429806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-proc\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.430072 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.430056    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klgzh\" (UniqueName: \"kubernetes.io/projected/d9594077-9d53-4a80-8bc3-088482851007-kube-api-access-klgzh\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.430232 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.430216    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-sys\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.430359 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.430347    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-lib-modules\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531108 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531076    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-proc\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531315 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klgzh\" (UniqueName: \"kubernetes.io/projected/d9594077-9d53-4a80-8bc3-088482851007-kube-api-access-klgzh\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531315 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531192    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-sys\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531315 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531220    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-lib-modules\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531315 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531227    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-proc\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531511 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531315    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-podres\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531511 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531330    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-sys\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531511 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531386    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-lib-modules\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.531511 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.531441    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9594077-9d53-4a80-8bc3-088482851007-podres\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.545574 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.545535    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgzh\" (UniqueName: \"kubernetes.io/projected/d9594077-9d53-4a80-8bc3-088482851007-kube-api-access-klgzh\") pod \"perf-node-gather-daemonset-4jhhz\" (UID: \"d9594077-9d53-4a80-8bc3-088482851007\") " pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.654055 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.653951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:39.837880 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:39.837847    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"]
Jul 06 15:05:39.840830 ip-10-0-140-204 kubenswrapper[2570]: W0706 15:05:39.840801    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd9594077_9d53_4a80_8bc3_088482851007.slice/crio-6c88c41e66877db721a13a52effdd9c587dc6f35e99a80a04d836c927a922f50 WatchSource:0}: Error finding container 6c88c41e66877db721a13a52effdd9c587dc6f35e99a80a04d836c927a922f50: Status 404 returned error can't find the container with id 6c88c41e66877db721a13a52effdd9c587dc6f35e99a80a04d836c927a922f50
Jul 06 15:05:40.432341 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:40.432314    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-qdh88_142f3bd6-dee6-406f-b7d9-3f7da5786619/networking-console-plugin/0.log"
Jul 06 15:05:40.570291 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:40.570239    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz" event={"ID":"d9594077-9d53-4a80-8bc3-088482851007","Type":"ContainerStarted","Data":"1781f3a0e2cd3dfa98a7f5d03376985d8c43a6480174fe6a0801337ac5c79ce4"}
Jul 06 15:05:40.570496 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:40.570320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz" event={"ID":"d9594077-9d53-4a80-8bc3-088482851007","Type":"ContainerStarted","Data":"6c88c41e66877db721a13a52effdd9c587dc6f35e99a80a04d836c927a922f50"}
Jul 06 15:05:40.571195 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:40.571170    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:40.594458 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:40.594398    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz" podStartSLOduration=1.594378117 podStartE2EDuration="1.594378117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:05:40.591982889 +0000 UTC m=+2490.800791298" watchObservedRunningTime="2026-07-06 15:05:40.594378117 +0000 UTC m=+2490.803186507"
Jul 06 15:05:41.035967 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:41.035928    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/2.log"
Jul 06 15:05:41.040730 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:41.040707    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pw8fj_239a5b97-8702-4a74-a4d5-fdad207c183a/console-operator/3.log"
Jul 06 15:05:41.528618 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:41.528587    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-647d98b875-85hnl_f780604b-3155-425a-aec0-abfb026dd718/console/0.log"
Jul 06 15:05:43.425085 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:43.425056    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vls8r_ad56e44f-e089-427b-9532-00d941fa3b80/dns/0.log"
Jul 06 15:05:43.456276 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:43.456231    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vls8r_ad56e44f-e089-427b-9532-00d941fa3b80/kube-rbac-proxy/0.log"
Jul 06 15:05:43.513890 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:43.513864    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b9cph_fc515b02-bb79-4f8d-9122-ca6043180ae8/dns-node-resolver/0.log"
Jul 06 15:05:44.222137 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:44.222098    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6vdx_c6ed3cfe-e58e-4b15-ad73-2c821535d27d/node-ca/0.log"
Jul 06 15:05:45.458299 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:45.458276    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d848d496-gfwqq_151c4ae4-f7fe-4bdd-8cd6-3919ded59618/kube-auth-proxy/0.log"
Jul 06 15:05:46.485082 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:46.485051    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z7n64_29f5a40d-93f2-434a-8da5-624bdbd9b4cc/serve-healthcheck-canary/0.log"
Jul 06 15:05:47.010386 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.010355    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-sg7jt_6dda94dd-e4ce-4c50-bb3c-211940025ef0/insights-operator/0.log"
Jul 06 15:05:47.010685 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.010670    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-sg7jt_6dda94dd-e4ce-4c50-bb3c-211940025ef0/insights-operator/1.log"
Jul 06 15:05:47.219552 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.219528    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wnznz_6c13e903-7cad-423a-b8e7-48a73bc2b241/kube-rbac-proxy/0.log"
Jul 06 15:05:47.245302 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.245270    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wnznz_6c13e903-7cad-423a-b8e7-48a73bc2b241/exporter/0.log"
Jul 06 15:05:47.270628 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.270552    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wnznz_6c13e903-7cad-423a-b8e7-48a73bc2b241/extractor/0.log"
Jul 06 15:05:47.595702 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:47.595671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8frwf/perf-node-gather-daemonset-4jhhz"
Jul 06 15:05:50.438834 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:50.438802    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d848d496-gfwqq_151c4ae4-f7fe-4bdd-8cd6-3919ded59618/kube-auth-proxy/0.log"
Jul 06 15:05:50.482414 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:50.482288    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mkl2l_95cc674f-1537-4e2a-b4b6-ddf361f4b5ff/server/0.log"
Jul 06 15:05:50.575176 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:50.575101    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kb6f4_ae0395e6-8234-4c18-9476-b9929fadc979/manager/0.log"
Jul 06 15:05:50.752664 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:50.752573    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-flqn8_ab11157e-a292-4369-bf5c-be0013bfaef2/manager/0.log"
Jul 06 15:05:50.772075 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:50.772050    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9447c645c-flqn8_ab11157e-a292-4369-bf5c-be0013bfaef2/copy-manifests/0.log"
Jul 06 15:05:52.022080 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:52.022045    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d848d496-gfwqq_151c4ae4-f7fe-4bdd-8cd6-3919ded59618/kube-auth-proxy/0.log"
Jul 06 15:05:52.284276 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:52.284249    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b6868f88d-n58v5_11f5157f-f23c-4067-a47f-e947fc5dc36b/manager/0.log"
Jul 06 15:05:57.372467 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:57.372432    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9ktb2_da2dfe16-32aa-421d-b094-4dddab657b2f/migrator/0.log"
Jul 06 15:05:57.395868 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:57.395819    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9ktb2_da2dfe16-32aa-421d-b094-4dddab657b2f/graceful-termination/0.log"
Jul 06 15:05:57.838100 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:57.838062    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-98pgc_9a37022f-52f7-46d1-ba09-448059337885/kube-storage-version-migrator-operator/1.log"
Jul 06 15:05:57.839013 ip-10-0-140-204 kubenswrapper[2570]: I0706 15:05:57.838977    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-98pgc_9a37022f-52f7-46d1-ba09-448059337885/kube-storage-version-migrator-operator/0.log"