Jul 21 00:14:49.907467 ip-10-0-140-105 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 00:14:49.907483 ip-10-0-140-105 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 00:14:49.907491 ip-10-0-140-105 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 00:14:49.907804 ip-10-0-140-105 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 00:15:00.052805 ip-10-0-140-105 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 00:15:00.052823 ip-10-0-140-105 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eb23c100bdd8481e823c32d5bb065e75 --
Jul 21 00:16:40.187612 ip-10-0-140-105 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 00:16:40.626677 ip-10-0-140-105 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 00:16:40.626677 ip-10-0-140-105 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 00:16:40.626677 ip-10-0-140-105 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 00:16:40.626677 ip-10-0-140-105 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 00:16:40.626677 ip-10-0-140-105 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 00:16:40.627764 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.627679    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633413    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633428    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633433    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633436    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633439    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.633433 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633442    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633445    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633448    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633451    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633455    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633457    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633460    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633463    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633465    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633468    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633471    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633473    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633476    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633479    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633489    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633492    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633494    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633497    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633499    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.633648 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633502    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633507    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633511    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633514    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633517    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633520    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633522    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633526    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633528    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633531    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633533    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633536    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633538    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633542    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633544    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633548    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633551    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633555    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633559    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.634098 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633562    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633564    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633567    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633570    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633572    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633575    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633578    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633580    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633584    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633587    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633589    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633592    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633595    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633597    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633600    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633602    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633605    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633607    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633610    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633613    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.634583 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633615    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633618    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633620    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633623    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633626    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633630    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633633    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633636    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633639    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633642    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633645    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633648    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633650    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633653    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633656    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633660    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633663    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633665    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633668    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.635067 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633671    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633673    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633676    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.633679    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634540    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634545    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634548    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634551    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634553    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634556    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634558    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634561    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634564    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634566    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634569    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634572    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.634575    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635333    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635340    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635343    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.635578 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635346    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635349    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635352    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635354    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635357    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635359    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635362    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635365    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635367    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635370    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635372    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635375    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635377    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635380    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635383    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635386    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635389    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635392    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635395    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635397    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.636047 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635400    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635403    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635405    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635408    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635411    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635414    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635416    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635419    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635421    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635423    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635426    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635429    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635431    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635434    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635436    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635439    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635442    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635448    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635452    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635455    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.636557 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635458    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635461    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635465    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635469    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635472    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635475    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635478    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635481    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635484    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635488    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635491    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635494    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635497    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635500    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635502    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635505    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635507    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635510    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635513    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.637058 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635516    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635518    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635521    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635523    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635526    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635529    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635532    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635534    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635537    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635540    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635543    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635546    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635548    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635551    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635553    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635556    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635559    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635561    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635565    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635567    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.637608 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635570    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635572    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635575    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635578    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635581    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635584    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635586    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635589    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635592    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635594    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635597    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635599    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635603    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635607    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635610    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635613    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635616    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635972    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635977    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635980    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.638463 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635983    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635986    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635990    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635993    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635996    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.635999    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636002    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636005    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636007    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636010    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636013    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636015    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636018    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636021    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636023    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636026    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636029    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636032    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636034    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.639155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636037    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636039    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636042    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636045    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636047    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636050    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636052    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636055    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636058    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636060    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636063    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636065    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636068    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636071    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636073    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636076    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636079    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636082    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636085    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636087    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.639653 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636090    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636093    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636095    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636098    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636101    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636104    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636106    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636108    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636111    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636114    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636117    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636119    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636122    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636124    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636127    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636130    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636132    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636134    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636137    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636140    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.640147 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636142    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636145    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636147    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636149    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636152    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636154    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636157    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636162    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636165    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636167    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636170    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636173    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636175    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636178    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636181    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636185    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636189    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636192    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636195    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636198    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.640661 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636200    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636203    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636206    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636208    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636211    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636213    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636216    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636218    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636221    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636223    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636226    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636228    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636232    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636236    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636239    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636242    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636244    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636341    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636363    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636373    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636378    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 00:16:40.641167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636382    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636386    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636390    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636394    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636398    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636401    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636405    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636409    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636412    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636415    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636418    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636421    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636424    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636427    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636432    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636435    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636438    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636441    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636444    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636448    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636451    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636454    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636458    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636461    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 00:16:40.641697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636464    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636467    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636470    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636473    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636477    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636480    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636484    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636487    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636490    2561 flags.go:64] FLAG: --enable-server="true"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636493    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636498    2561 flags.go:64] FLAG: --event-burst="100"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636501    2561 flags.go:64] FLAG: --event-qps="50"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636504    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636507    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636510    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636514    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636517    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636520    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636524    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636527    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636530    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636533    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636536    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636540    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636543    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 00:16:40.642260 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636546    2561 flags.go:64] FLAG: --feature-gates=""
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636550    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636553    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636556    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636559    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636563    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636566    2561 flags.go:64] FLAG: --help="false"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636569    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636572    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636575    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636578    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636581    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636584    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636588    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636590    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636593    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636596    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636599    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636603    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636607    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636610    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636613    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636616    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636619    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 00:16:40.642921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636621    2561 flags.go:64] FLAG: --lock-file=""
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636624    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636627    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636630    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636644    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636647    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636650    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636654    2561 flags.go:64] FLAG: --logging-format="text"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636657    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636660    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636663    2561 flags.go:64] FLAG: --manifest-url=""
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636666    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636671    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636674    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636678    2561 flags.go:64] FLAG: --max-pods="110"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636681    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636684    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636687    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636690    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636693    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636696    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636699    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636706    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636709    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636712    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 00:16:40.643531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636715    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636718    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636725    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636728    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636731    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636734    2561 flags.go:64] FLAG: --port="10250"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636737    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636741    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0edfcdbe6d35372bd"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636744    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636747    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636750    2561 flags.go:64] FLAG: --register-node="true"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636753    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636757    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636760    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636763    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636767    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636770    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636773    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636777    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636780    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636783    2561 flags.go:64] FLAG: --runonce="false"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636785    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636789    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636792    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636795    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 00:16:40.644182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636798    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636801    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636804    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636807    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636810    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636813    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636816    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636819    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636822    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636827    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636830    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636835    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636837    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636840    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636844    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636847    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636851    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636853    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636856    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636859    2561 flags.go:64] FLAG: --v="2"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636863    2561 flags.go:64] FLAG: --version="false"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636867    2561 flags.go:64] FLAG: --vmodule=""
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636871    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.636875    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636992    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.645330 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636996    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.636999    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637002    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637004    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637007    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637009    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637012    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637014    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637017    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637019    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637022    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637025    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637027    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637030    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637032    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637035    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637039    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637043    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637047    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637049    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.646462 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637052    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637055    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637057    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637062    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637066    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637069    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637071    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637074    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637077    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637080    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637083    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637086    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637090    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637094    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637097    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637101    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637104    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637106    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637109    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.646971 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637111    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637114    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637116    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637119    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637121    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637124    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637126    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637129    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637131    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637135    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637138    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637140    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637143    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637145    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637148    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637150    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637154    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637157    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637159    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.647533 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637162    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637164    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637167    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637169    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637172    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637175    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637183    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637186    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637189    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637191    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637194    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637196    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637198    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637201    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637204    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637206    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637209    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637212    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637214    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637217    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.648371 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637220    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637222    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637226    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637229    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637231    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637234    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637236    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637239    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637242    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637245    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637247    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637250    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637253    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637255    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637258    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637260    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637263    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637266    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637270    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.649081 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.637273    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.637279    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.646408    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.646426    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.647354    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.647367    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647450    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647459    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647465    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647469    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647474    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647478    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647482    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647487    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.649605 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647492    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647496    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647501    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647505    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647509    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647513    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647517    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647521    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647525    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647529    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647533    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647537    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647543    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647549    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647553    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647558    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647562    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647568    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647572    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647577    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.650133 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647581    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647586    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647591    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647595    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647599    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647604    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647608    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647613    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647617    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647621    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647625    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647630    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647634    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647638    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647642    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647647    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647651    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647656    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647661    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647665    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.650646 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647670    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647675    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647679    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647683    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647690    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647696    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647702    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647707    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647711    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647717    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647721    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647726    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647730    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647734    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647738    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647743    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647747    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647751    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647755    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647759    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.651142 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647763    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647768    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647772    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647776    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647780    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647784    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647788    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647793    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647797    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647801    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647805    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647811    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647815    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647819    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647823    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647827    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647832    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647836    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647841    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647845    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.651723 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647849    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647853    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647857    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647862    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647866    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647872    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647878    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647883    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647887    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647892    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.647896    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.647904    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648096    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648104    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 00:16:40.652192 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648109    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648114    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648119    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648123    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648128    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648132    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648138    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648146    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648153    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648159    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648164    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648169    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648174    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648178    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648183    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648187    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648192    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648197    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648201    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 00:16:40.652702 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648206    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648210    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648214    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648218    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648222    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648227    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648231    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648236    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648240    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648244    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648248    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648252    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648256    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648260    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648265    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648269    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648273    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648277    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648281    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648286    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 00:16:40.653414 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648290    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648294    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648299    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648303    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648327    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648332    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648336    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648340    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648344    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648349    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648353    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648357    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648361    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648365    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648369    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648373    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648377    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648381    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648384    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648388    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 00:16:40.653902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648392    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648396    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648400    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648405    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648409    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648413    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648417    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648422    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648426    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648432    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648438    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648443    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648448    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648453    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648458    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648462    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648467    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648472    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648477    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 00:16:40.654491 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648481    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648486    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648490    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648494    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648498    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648502    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648507    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648511    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648515    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648519    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648523    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648527    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648531    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648536    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648540    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648544    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648548    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648552    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 00:16:40.655045 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:40.648556    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 00:16:40.655514 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.648564    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 00:16:40.655514 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.648745    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 00:16:40.655514 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.654881    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 00:16:40.656607 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.656594    2561 server.go:1019] "Starting client certificate rotation"
Jul 21 00:16:40.656727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.656709    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 00:16:40.656765 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.656746    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 00:16:40.659427 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.659410    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 00:16:40.660602 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.660585    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 00:16:40.677943 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.677853    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 21 00:16:40.681240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.681220    2561 log.go:25] "Validated CRI v1 image API"
Jul 21 00:16:40.682068 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.682052    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 00:16:40.685517 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.685494    2561 fs.go:135] Filesystem UUIDs: map[5b9a3258-bdef-4c35-8b3b-d2c4bd87df2c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 aaf997cd-c7e5-48c5-ab68-771a53fdc301:/dev/nvme0n1p4]
Jul 21 00:16:40.685579 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.685515    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 00:16:40.692140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.692016    2561 manager.go:216] Machine: {Timestamp:2026-07-21 00:16:40.690592838 +0000 UTC m=+0.375583938 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098221 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273cb4276a95df2e3c6e703a3eaee6 SystemUUID:ec273cb4-276a-95df-2e3c-6e703a3eaee6 BootID:eb23c100-bdd8-481e-823c-32d5bb065e75 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:95:10:4b:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:95:10:4b:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:4b:ea:15:32:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 00:16:40.692140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.692128    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 00:16:40.692271 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.692196    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 00:16:40.692271 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.692259    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 00:16:40.695767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.695740    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 00:16:40.695900 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.695769    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 00:16:40.696628 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.696618    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 00:16:40.696667 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.696629    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 00:16:40.696667 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.696646    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 00:16:40.698304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.698293    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 00:16:40.700222 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.700212    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 00:16:40.704974 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.704962    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 00:16:40.705079 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705070    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 00:16:40.705115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705097    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 00:16:40.705115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705107    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 00:16:40.705168 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705125    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 00:16:40.705168 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705135    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 21 00:16:40.705168 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705146    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 00:16:40.705575 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705558    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 00:16:40.705638 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.705580    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 00:16:40.707904 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.707888    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 00:16:40.710283 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.710270    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 00:16:40.710360 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.710300    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 00:16:40.711669 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711659    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711675    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711682    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711687    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711706    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711713    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711721    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 00:16:40.711727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711726    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 00:16:40.711895 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711733    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 00:16:40.711895 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711739    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 00:16:40.711895 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711750    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 00:16:40.711895 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.711756    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 00:16:40.712683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.712673    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 00:16:40.712683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.712682    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 00:16:40.715791 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.715776    2561 server.go:1268] "Started kubelet"
Jul 21 00:16:40.715919 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.715869    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 00:16:40.715975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.715963    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 00:16:40.716032 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.715984    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 00:16:40.716593 ip-10-0-140-105 systemd[1]: Started Kubernetes Kubelet.
Jul 21 00:16:40.716767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.716752    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 00:16:40.717208 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.717051    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 00:16:40.717208 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.717051    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 00:16:40.718231 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.718212    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 00:16:40.718601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.718587    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 00:16:40.721526 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.720404    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-105.ec2.internal.18c425e1044f9050  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-105.ec2.internal,UID:ip-10-0-140-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-105.ec2.internal,},FirstTimestamp:2026-07-21 00:16:40.715751504 +0000 UTC m=+0.400742589,LastTimestamp:2026-07-21 00:16:40.715751504 +0000 UTC m=+0.400742589,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-105.ec2.internal,}"
Jul 21 00:16:40.722797 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.722781    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 00:16:40.727236 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.727219    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 00:16:40.727298 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.727220    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.727956    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.728401    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.728428    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.728585    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.728734    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.728741    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729363    2561 factory.go:153] Registering CRI-O factory
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729378    2561 factory.go:223] Registration of the crio container factory successfully
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729430    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729440    2561 factory.go:55] Registering systemd factory
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729449    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729471    2561 factory.go:103] Registering Raw factory
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729484    2561 manager.go:1196] Started watching for new ooms in manager
Jul 21 00:16:40.730507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.729931    2561 manager.go:349] Starting recovery of all containers
Jul 21 00:16:40.733509 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.733483    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tprjb"
Jul 21 00:16:40.738365 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.738332    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 00:16:40.739862 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.739837    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 00:16:40.740000 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.739976    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 00:16:40.741105 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.741084    2561 manager.go:354] Recovery completed
Jul 21 00:16:40.741196 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.741101    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tprjb"
Jul 21 00:16:40.745575 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.745559    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.747747 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.747733    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.747804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.747764    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.747804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.747777    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.748293 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.748277    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 00:16:40.748293 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.748290    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 00:16:40.748419 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.748320    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 00:16:40.748419 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.748350    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.749037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.749022    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.749076 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.749050    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.749076 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.749059    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.750382 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.750371    2561 policy_none.go:49] "None policy: Start"
Jul 21 00:16:40.750430 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.750386    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 00:16:40.750430 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.750397    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 00:16:40.752379 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.752367    2561 policy_none.go:47] "Start"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.789778    2561 manager.go:337] "Starting Device Plugin manager"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.789970    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.789984    2561 server.go:87] "Starting device plugin registration server"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.790273    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.790286    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.790378    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.790481    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.790493    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.791038    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 00:16:40.813052 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.791100    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:40.839151 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.839131    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 00:16:40.839268 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.839169    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 00:16:40.839268 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.839188    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 00:16:40.839268 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.839216    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 00:16:40.841795 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.841778    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 00:16:40.891407 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.891325    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.892162 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.892148    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.892240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.892180    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.892240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.892194    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.892240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.892223    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.901631 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.901609    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.901719 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.901635    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-105.ec2.internal\": node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:40.926223 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.926199    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:40.939670 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.939652    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"]
Jul 21 00:16:40.939750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.939712    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.941051 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.941037    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.941121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.941067    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.941121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.941078    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.943958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.943808    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.943958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.943901    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.944075 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.943970    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.946238 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946209    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.946342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946246    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.946342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946261    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.946342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946264    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.946342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946289    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.946342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.946304    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.948547 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.948533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.948607 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.948559    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 00:16:40.949156 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.949143    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 00:16:40.949217 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.949168    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 00:16:40.949217 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:40.949183    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Jul 21 00:16:40.971805 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.971785    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-105.ec2.internal\" not found" node="ip-10-0-140-105.ec2.internal"
Jul 21 00:16:40.976040 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:40.976025    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-105.ec2.internal\" not found" node="ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.027121 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.027101    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.030425 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.030410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.030491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.030434    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.030491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.030461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e952599a926de01ba0ecf526bfb18c75-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"e952599a926de01ba0ecf526bfb18c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.127715 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.127668    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.130925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.130905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.130978 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.130933    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e952599a926de01ba0ecf526bfb18c75-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"e952599a926de01ba0ecf526bfb18c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.130978 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.130950    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.131052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.130993    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.131052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.131015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/469ca48e4a79de685533b39875a7f059-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"469ca48e4a79de685533b39875a7f059\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.131052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.131018    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e952599a926de01ba0ecf526bfb18c75-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"e952599a926de01ba0ecf526bfb18c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.228438 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.228371    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.274774 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.274744    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.278365 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.278346    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.328926 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.328896    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.429342 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.429300    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.529823 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.529760    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.630215 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.630182    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.655512 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.655487    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 00:16:41.655648 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.655616    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 00:16:41.730547 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:41.730521    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Jul 21 00:16:41.732566 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.732551    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 00:16:41.746993 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.746967    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 00:11:41 +0000 UTC" deadline="2028-01-21 03:23:27.070677289 +0000 UTC"
Jul 21 00:16:41.746993 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.746990    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13179h6m45.323689818s"
Jul 21 00:16:41.817039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.817017    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 00:16:41.828982 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.828952    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.844964 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.844945    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 00:16:41.846296 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.846278    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Jul 21 00:16:41.854704 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.854684    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 00:16:41.940917 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:41.940890    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode952599a926de01ba0ecf526bfb18c75.slice/crio-3bf18cda27b55d225825d17d4b4efa339e87b2ac4399fada406465fd3525b1ed WatchSource:0}: Error finding container 3bf18cda27b55d225825d17d4b4efa339e87b2ac4399fada406465fd3525b1ed: Status 404 returned error can't find the container with id 3bf18cda27b55d225825d17d4b4efa339e87b2ac4399fada406465fd3525b1ed
Jul 21 00:16:41.941417 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:41.941397    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469ca48e4a79de685533b39875a7f059.slice/crio-1fc69aed49a235e022b6ca53fe5258874114fa3598bae91e9202e7859988c13a WatchSource:0}: Error finding container 1fc69aed49a235e022b6ca53fe5258874114fa3598bae91e9202e7859988c13a: Status 404 returned error can't find the container with id 1fc69aed49a235e022b6ca53fe5258874114fa3598bae91e9202e7859988c13a
Jul 21 00:16:41.948177 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.948159    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 00:16:41.993049 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:41.993026    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 00:16:43.675163 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.675134    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 00:16:43.807651 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.807574    2561 apiserver.go:52] "Watching apiserver"
Jul 21 00:16:43.816674 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.816641    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 00:16:43.817194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.817173    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-67hqq","openshift-ovn-kubernetes/ovnkube-node-8qdxc","kube-system/global-pull-secret-syncer-t6z2j","kube-system/konnectivity-agent-lqp96","kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx","openshift-image-registry/node-ca-kpx88","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal","openshift-multus/multus-additional-cni-plugins-hhlpr","openshift-multus/multus-vzcbb","openshift-cluster-node-tuning-operator/tuned-c4924","openshift-dns/node-resolver-m7f5t","openshift-multus/network-metrics-daemon-987l2","openshift-network-diagnostics/network-check-target-j4jz9"]
Jul 21 00:16:43.819733 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.819712    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.822030 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.822001    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbhtq\""
Jul 21 00:16:43.822539 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.822301    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 00:16:43.822539 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.822350    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.822539 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.822306    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.823359 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.822911    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.826014 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.825474    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.826014 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:43.825643    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.826557    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.826788    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m4s68\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.827107    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.827335    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.827580    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.828615    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.829060    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.830720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.829691    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 00:16:43.832345 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.832297    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.832778 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.832486    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.834979 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.834960    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:43.835077 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.834988    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.835574 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.835546    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 00:16:43.835754 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.835737    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 00:16:43.835839 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.835827    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.836072 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.836054    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-22jnk\""
Jul 21 00:16:43.836164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.836149    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 00:16:43.838213 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838192    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ncz79\""
Jul 21 00:16:43.838313 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838299    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 00:16:43.838362 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838317    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.838519 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838503    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8hjml\""
Jul 21 00:16:43.838589 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-log-socket\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838646 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-registration-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.838646 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838615    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-systemd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838743 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838648    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-bin\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838743 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838674    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-env-overrides\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838743 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838698    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-konnectivity-ca\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.838743 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-sys-fs\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-netns\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838786    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-etc-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-dbus\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838860    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhfx\" (UniqueName: \"kubernetes.io/projected/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-kube-api-access-hvhfx\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.838930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-device-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838932    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-ovn\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.838973    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-node-log\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839049    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839039    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-netd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkdmw\" (UniqueName: \"kubernetes.io/projected/f88e2920-094e-4b3d-b970-2267edefac10-kube-api-access-rkdmw\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-slash\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839234    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-agent-certs\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-socket-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-host-slash\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-var-lib-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-script-lib\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839396    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wtln\" (UniqueName: \"kubernetes.io/projected/2d686cd2-f11a-4e5c-b805-2620d608e3fc-kube-api-access-7wtln\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839450    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-kubelet-config\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839489    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-systemd-units\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839591    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-config\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839602    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovn-node-metrics-cert\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.839695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839672    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-iptables-alerter-script\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.840089 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.839710    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-kubelet\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.840628 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.840469    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pqj8k\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.840867    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.841170    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.841424    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.841662    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.841842    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.841856    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 00:16:43.842061 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.842024    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.843782 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.843764    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.844422 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.844405    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h49pd\""
Jul 21 00:16:43.844531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.844432    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 00:16:43.847302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.847279    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:43.849669 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.849650    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.849785 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.849767    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cnhz2\""
Jul 21 00:16:43.849902 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.849886    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 00:16:43.850051 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.850031    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.850161 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.850148    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jwmlj\""
Jul 21 00:16:43.850318 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.850302    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 00:16:43.850393 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.850379    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:43.850493 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:43.850472    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:43.851946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.851930    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:43.852035 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:43.851989    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:43.863019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.862978    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4hp5q"
Jul 21 00:16:43.870428 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.870406    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4hp5q"
Jul 21 00:16:43.932019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.931995    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 00:16:43.940485 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940460    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-conf\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940497    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-systemd\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940521    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-tuned\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940567    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-multus-certs\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940598    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-etc-kubernetes\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.940637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-iptables-alerter-script\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940652    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-kubelet\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-log-socket\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-registration-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940721    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-lib-modules\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940744    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hw2l\" (UniqueName: \"kubernetes.io/projected/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-kube-api-access-5hw2l\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940777    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-systemd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-env-overrides\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysconfig\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-run\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-tmp\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940863    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cnibin\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-netns\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.940913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940933    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-sys\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhfx\" (UniqueName: \"kubernetes.io/projected/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-kube-api-access-hvhfx\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.940983    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-device-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a151b2d-b542-4a6b-99c7-5ce86b3be500-host\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-tmp-dir\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941049    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-system-cni-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941075    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-node-log\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941134    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkskn\" (UniqueName: \"kubernetes.io/projected/8a151b2d-b542-4a6b-99c7-5ce86b3be500-kube-api-access-gkskn\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941168    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-socket-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941214    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-host\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941425    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-bin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-hostroot\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941529    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wtln\" (UniqueName: \"kubernetes.io/projected/2d686cd2-f11a-4e5c-b805-2620d608e3fc-kube-api-access-7wtln\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.941582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941561    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-kubelet-config\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941701    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-config\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovn-node-metrics-cert\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941773    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-modprobe-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941801    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-os-release\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-system-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-cnibin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941894    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-cni-binary-copy\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941921    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-socket-dir-parent\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.941979    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-kubelet\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.942371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.942009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm4gb\" (UniqueName: \"kubernetes.io/projected/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-kube-api-access-nm4gb\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:43.942910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.942743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-iptables-alerter-script\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.942910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.942836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-kubelet\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.942910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.942892    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-log-socket\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.943055 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.942959    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-registration-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.943055 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.943043    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-systemd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.943534 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.943513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-env-overrides\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.943656 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.943633    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-netns\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.943782 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.943766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.944166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-device-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.944258 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-node-log\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.944320 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.944405 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.944531 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944515    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-socket-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.944864 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.944841    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-kubelet-config\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.945350 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.945329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-config\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.945691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.945669    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 00:16:43.946820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946792    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-bin\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.946899 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946851    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-konnectivity-ca\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.946899 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946878    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-bin\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.946899 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946892    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-sys-fs\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.947044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946935    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-k8s-cni-cncf-io\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.947044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.946972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-netns\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.947044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947002    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-multus-daemon-config\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.947044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-etc-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.947220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.947220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947093    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-dbus\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.947220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a151b2d-b542-4a6b-99c7-5ce86b3be500-serviceca\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:43.947220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.947220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-kubernetes\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-var-lib-kubelet\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4gqp\" (UniqueName: \"kubernetes.io/projected/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-kube-api-access-h4gqp\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947307    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-multus\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-conf-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-ovn\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.947415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947394    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-netd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947424    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947473    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-konnectivity-ca\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-sys-fs\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947475    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkdmw\" (UniqueName: \"kubernetes.io/projected/f88e2920-094e-4b3d-b970-2267edefac10-kube-api-access-rkdmw\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-etc-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.947752 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrnb8\" (UniqueName: \"kubernetes.io/projected/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-kube-api-access-lrnb8\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:43.948038 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947786    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948038 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.947903    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-dbus\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.948120 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948050    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-run-ovn\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948120 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-cni-netd\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948176 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948159    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e2920-094e-4b3d-b970-2267edefac10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.948221 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948203    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.948267 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948253    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-slash\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-agent-certs\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.948365 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-host-slash\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948430 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948413    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-hosts-file\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:43.948528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948501    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.948585 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948546    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:43.948636 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsv5s\" (UniqueName: \"kubernetes.io/projected/81869e75-484e-4145-b3d0-1a9261042b4f-kube-api-access-hsv5s\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.948674 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948639    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-host-slash\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.948674 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-var-lib-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948745 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-script-lib\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948797 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948767    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:43.948797 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948793    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-os-release\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:43.948890 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948809    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-systemd-units\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.948890 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.948886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-systemd-units\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.949276 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.949259    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovnkube-script-lib\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.949334 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.949321    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-host-slash\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.949379 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.949352    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d686cd2-f11a-4e5c-b805-2620d608e3fc-var-lib-openvswitch\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.953833 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:43.949494    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:43.953833 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:43.949580    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:16:44.449541792 +0000 UTC m=+3.033404671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:43.953833 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.950060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d686cd2-f11a-4e5c-b805-2620d608e3fc-ovn-node-metrics-cert\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:43.957512 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.953982    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"469ca48e4a79de685533b39875a7f059","Type":"ContainerStarted","Data":"1fc69aed49a235e022b6ca53fe5258874114fa3598bae91e9202e7859988c13a"}
Jul 21 00:16:43.957512 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.955699    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkdmw\" (UniqueName: \"kubernetes.io/projected/f88e2920-094e-4b3d-b970-2267edefac10-kube-api-access-rkdmw\") pod \"aws-ebs-csi-driver-node-d5hlx\" (UID: \"f88e2920-094e-4b3d-b970-2267edefac10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:43.957512 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.956643    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" event={"ID":"e952599a926de01ba0ecf526bfb18c75","Type":"ContainerStarted","Data":"3bf18cda27b55d225825d17d4b4efa339e87b2ac4399fada406465fd3525b1ed"}
Jul 21 00:16:43.962012 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.961993    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3c649de-ff1c-4205-9d5e-eef443b6cc3b-agent-certs\") pod \"konnectivity-agent-lqp96\" (UID: \"f3c649de-ff1c-4205-9d5e-eef443b6cc3b\") " pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:43.963629 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.963601    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhfx\" (UniqueName: \"kubernetes.io/projected/783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f-kube-api-access-hvhfx\") pod \"iptables-alerter-67hqq\" (UID: \"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f\") " pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:43.964919 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:43.964898    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wtln\" (UniqueName: \"kubernetes.io/projected/2d686cd2-f11a-4e5c-b805-2620d608e3fc-kube-api-access-7wtln\") pod \"ovnkube-node-8qdxc\" (UID: \"2d686cd2-f11a-4e5c-b805-2620d608e3fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:44.049862 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-modprobe-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049868    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-os-release\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049890    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-system-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049910    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-cnibin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-cni-binary-copy\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049952    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-socket-dir-parent\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049974    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049996    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-kubelet\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049997    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-system-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050012    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-modprobe-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.049998    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-os-release\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050017    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm4gb\" (UniqueName: \"kubernetes.io/projected/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-kube-api-access-nm4gb\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050075    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-k8s-cni-cncf-io\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-kubelet\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050102    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-netns\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-multus-daemon-config\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050138    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-socket-dir-parent\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-cnibin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a151b2d-b542-4a6b-99c7-5ce86b3be500-serviceca\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050181    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050203    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-k8s-cni-cncf-io\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050209    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-kubernetes\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050253    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-var-lib-kubelet\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4gqp\" (UniqueName: \"kubernetes.io/projected/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-kube-api-access-h4gqp\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-multus\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.050487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-conf-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050383    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrnb8\" (UniqueName: \"kubernetes.io/projected/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-kube-api-access-lrnb8\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050407    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050452    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-hosts-file\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050502    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050505    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-cni-binary-copy\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050526    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsv5s\" (UniqueName: \"kubernetes.io/projected/81869e75-484e-4145-b3d0-1a9261042b4f-kube-api-access-hsv5s\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050549    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-netns\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-os-release\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-multus\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-conf\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050614    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-systemd\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050618    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-tuned\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050673    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050700    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81869e75-484e-4145-b3d0-1a9261042b4f-multus-daemon-config\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.051248 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050723    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-multus-certs\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-etc-kubernetes\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050780    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-lib-modules\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050804    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hw2l\" (UniqueName: \"kubernetes.io/projected/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-kube-api-access-5hw2l\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050858    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysconfig\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050869    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050880    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a151b2d-b542-4a6b-99c7-5ce86b3be500-serviceca\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050917    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-run\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050943    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-hosts-file\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.050980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-conf-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051104    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-d\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-kubernetes\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051213    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-os-release\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051279    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-multus-cni-dir\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051300    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051334    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysconfig\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-tmp\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051367    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cnibin\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051372    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-etc-kubernetes\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-run\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051392    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-sys\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051419    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a151b2d-b542-4a6b-99c7-5ce86b3be500-host\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-tmp-dir\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-system-cni-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkskn\" (UniqueName: \"kubernetes.io/projected/8a151b2d-b542-4a6b-99c7-5ce86b3be500-kube-api-access-gkskn\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-lib-modules\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-host\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051554    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-cnibin\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-bin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-hostroot\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051603    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-var-lib-kubelet\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051660    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-sys\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051680    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051694    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a151b2d-b542-4a6b-99c7-5ce86b3be500-host\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-host\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.051769    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:44.052939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-var-lib-cni-bin\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.051820    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:16:44.551803583 +0000 UTC m=+3.135666461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051905    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-hostroot\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051944    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-sysctl-conf\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.052073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-system-cni-dir\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.052211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-tmp-dir\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.051339    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81869e75-484e-4145-b3d0-1a9261042b4f-host-run-multus-certs\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.053929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.052388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-systemd\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.054381 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.054062    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-tmp\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.055568 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.055544    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-etc-tuned\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.057248 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.057212    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:44.057248 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.057237    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:44.057248 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.057250    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:44.057453 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.057324    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:16:44.55730549 +0000 UTC m=+3.141168378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:44.062982 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.062926    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hw2l\" (UniqueName: \"kubernetes.io/projected/ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665-kube-api-access-5hw2l\") pod \"node-resolver-m7f5t\" (UID: \"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665\") " pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.064101 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.063667    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm4gb\" (UniqueName: \"kubernetes.io/projected/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-kube-api-access-nm4gb\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:44.064101 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.064066    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrnb8\" (UniqueName: \"kubernetes.io/projected/947a2b50-c63c-4e20-8e5a-04ac4ff69f0c-kube-api-access-lrnb8\") pod \"tuned-c4924\" (UID: \"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c\") " pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.064238 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.064108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsv5s\" (UniqueName: \"kubernetes.io/projected/81869e75-484e-4145-b3d0-1a9261042b4f-kube-api-access-hsv5s\") pod \"multus-vzcbb\" (UID: \"81869e75-484e-4145-b3d0-1a9261042b4f\") " pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.064520 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.064425    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkskn\" (UniqueName: \"kubernetes.io/projected/8a151b2d-b542-4a6b-99c7-5ce86b3be500-kube-api-access-gkskn\") pod \"node-ca-kpx88\" (UID: \"8a151b2d-b542-4a6b-99c7-5ce86b3be500\") " pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.065237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.065215    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4gqp\" (UniqueName: \"kubernetes.io/projected/3a312e9c-35ec-4ea9-ac0e-f25cf30d3938-kube-api-access-h4gqp\") pod \"multus-additional-cni-plugins-hhlpr\" (UID: \"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938\") " pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.142036 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.142005    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67hqq"
Jul 21 00:16:44.149788 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.149567    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:16:44.156695 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.156672    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783fe8c4_9e4b_42ef_82d0_7f71a93bfb0f.slice/crio-b4aa2ae5e5365ceb18160425259956147cc42cca9e2ca346299d7373e5f92896 WatchSource:0}: Error finding container b4aa2ae5e5365ceb18160425259956147cc42cca9e2ca346299d7373e5f92896: Status 404 returned error can't find the container with id b4aa2ae5e5365ceb18160425259956147cc42cca9e2ca346299d7373e5f92896
Jul 21 00:16:44.158568 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.158533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx"
Jul 21 00:16:44.158795 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.158779    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d686cd2_f11a_4e5c_b805_2620d608e3fc.slice/crio-55689a64abaab67a8d9a194dbc4cd4fe48f2632cd147ce337932ed7897222cd5 WatchSource:0}: Error finding container 55689a64abaab67a8d9a194dbc4cd4fe48f2632cd147ce337932ed7897222cd5: Status 404 returned error can't find the container with id 55689a64abaab67a8d9a194dbc4cd4fe48f2632cd147ce337932ed7897222cd5
Jul 21 00:16:44.166100 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.166082    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:44.167155 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.167138    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88e2920_094e_4b3d_b970_2267edefac10.slice/crio-f2d89564de4a5f292763c8fc9a024630ab7f071044be01de7f975bbb6d2468ce WatchSource:0}: Error finding container f2d89564de4a5f292763c8fc9a024630ab7f071044be01de7f975bbb6d2468ce: Status 404 returned error can't find the container with id f2d89564de4a5f292763c8fc9a024630ab7f071044be01de7f975bbb6d2468ce
Jul 21 00:16:44.174231 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.174213    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c649de_ff1c_4205_9d5e_eef443b6cc3b.slice/crio-f8d44163a02d8ac798ec58bb1485350799ef92abbeece32be5cdc1eaec8a2f55 WatchSource:0}: Error finding container f8d44163a02d8ac798ec58bb1485350799ef92abbeece32be5cdc1eaec8a2f55: Status 404 returned error can't find the container with id f8d44163a02d8ac798ec58bb1485350799ef92abbeece32be5cdc1eaec8a2f55
Jul 21 00:16:44.174584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.174571    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kpx88"
Jul 21 00:16:44.182049 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.182030    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a151b2d_b542_4a6b_99c7_5ce86b3be500.slice/crio-9d8f0df5285aa28d8cc1c66f11b2363b9d86e19ca200e5d155ca53c848892b16 WatchSource:0}: Error finding container 9d8f0df5285aa28d8cc1c66f11b2363b9d86e19ca200e5d155ca53c848892b16: Status 404 returned error can't find the container with id 9d8f0df5285aa28d8cc1c66f11b2363b9d86e19ca200e5d155ca53c848892b16
Jul 21 00:16:44.188943 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.188927    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hhlpr"
Jul 21 00:16:44.198754 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.198731    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a312e9c_35ec_4ea9_ac0e_f25cf30d3938.slice/crio-bea92adcc9e584b7eb0f35bc8d6da21ba528ed40e5480acc741ed890d358f798 WatchSource:0}: Error finding container bea92adcc9e584b7eb0f35bc8d6da21ba528ed40e5480acc741ed890d358f798: Status 404 returned error can't find the container with id bea92adcc9e584b7eb0f35bc8d6da21ba528ed40e5480acc741ed890d358f798
Jul 21 00:16:44.199237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.199216    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vzcbb"
Jul 21 00:16:44.206811 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.206782    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c4924"
Jul 21 00:16:44.209080 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.209053    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81869e75_484e_4145_b3d0_1a9261042b4f.slice/crio-9e033ec388e5044e2a4106b7d4478eee361275cbde0f7591b0b9e2d34639983c WatchSource:0}: Error finding container 9e033ec388e5044e2a4106b7d4478eee361275cbde0f7591b0b9e2d34639983c: Status 404 returned error can't find the container with id 9e033ec388e5044e2a4106b7d4478eee361275cbde0f7591b0b9e2d34639983c
Jul 21 00:16:44.215226 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.215203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m7f5t"
Jul 21 00:16:44.217799 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.217777    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947a2b50_c63c_4e20_8e5a_04ac4ff69f0c.slice/crio-22fbe8092e1aae94c0cfa548919bf5df431ebd94539f7f73b69b35391de0382e WatchSource:0}: Error finding container 22fbe8092e1aae94c0cfa548919bf5df431ebd94539f7f73b69b35391de0382e: Status 404 returned error can't find the container with id 22fbe8092e1aae94c0cfa548919bf5df431ebd94539f7f73b69b35391de0382e
Jul 21 00:16:44.228538 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:16:44.228515    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7c1b1c_accb_4f4f_9b5e_be8b7c2a8665.slice/crio-8b1a2730fc2b344a353278725a854e9b93ad2a3a6ab29c91766de5843273d1c7 WatchSource:0}: Error finding container 8b1a2730fc2b344a353278725a854e9b93ad2a3a6ab29c91766de5843273d1c7: Status 404 returned error can't find the container with id 8b1a2730fc2b344a353278725a854e9b93ad2a3a6ab29c91766de5843273d1c7
Jul 21 00:16:44.453938 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.453855    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:44.454100 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.454004    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:44.454100 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.454060    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:16:45.454043894 +0000 UTC m=+4.037906773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:44.554276 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.554246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:44.554489 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.554396    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:44.554489 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.554482    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:16:45.554464756 +0000 UTC m=+4.138327631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:44.654777 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.654741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:44.654921 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.654901    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:44.654981 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.654926    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:44.654981 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.654940    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:44.655048 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:44.654997    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:16:45.654982231 +0000 UTC m=+4.238845120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:44.874595 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.874509    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 00:11:43 +0000 UTC" deadline="2028-02-22 05:13:11.70701297 +0000 UTC"
Jul 21 00:16:44.874595 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.874544    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13948h56m26.832473478s"
Jul 21 00:16:44.960158 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.960122    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m7f5t" event={"ID":"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665","Type":"ContainerStarted","Data":"8b1a2730fc2b344a353278725a854e9b93ad2a3a6ab29c91766de5843273d1c7"}
Jul 21 00:16:44.962197 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.962167    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kpx88" event={"ID":"8a151b2d-b542-4a6b-99c7-5ce86b3be500","Type":"ContainerStarted","Data":"9d8f0df5285aa28d8cc1c66f11b2363b9d86e19ca200e5d155ca53c848892b16"}
Jul 21 00:16:44.963271 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.963249    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lqp96" event={"ID":"f3c649de-ff1c-4205-9d5e-eef443b6cc3b","Type":"ContainerStarted","Data":"f8d44163a02d8ac798ec58bb1485350799ef92abbeece32be5cdc1eaec8a2f55"}
Jul 21 00:16:44.965146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.965121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c4924" event={"ID":"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c","Type":"ContainerStarted","Data":"22fbe8092e1aae94c0cfa548919bf5df431ebd94539f7f73b69b35391de0382e"}
Jul 21 00:16:44.967371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.967347    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vzcbb" event={"ID":"81869e75-484e-4145-b3d0-1a9261042b4f","Type":"ContainerStarted","Data":"9e033ec388e5044e2a4106b7d4478eee361275cbde0f7591b0b9e2d34639983c"}
Jul 21 00:16:44.970736 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.970688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerStarted","Data":"bea92adcc9e584b7eb0f35bc8d6da21ba528ed40e5480acc741ed890d358f798"}
Jul 21 00:16:44.971974 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.971950    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx" event={"ID":"f88e2920-094e-4b3d-b970-2267edefac10","Type":"ContainerStarted","Data":"f2d89564de4a5f292763c8fc9a024630ab7f071044be01de7f975bbb6d2468ce"}
Jul 21 00:16:44.973401 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.973336    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"55689a64abaab67a8d9a194dbc4cd4fe48f2632cd147ce337932ed7897222cd5"}
Jul 21 00:16:44.974763 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:44.974738    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67hqq" event={"ID":"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f","Type":"ContainerStarted","Data":"b4aa2ae5e5365ceb18160425259956147cc42cca9e2ca346299d7373e5f92896"}
Jul 21 00:16:45.462765 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.462717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:45.462948 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.462928    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:45.463036 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.462997    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:16:47.46297915 +0000 UTC m=+6.046842026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:45.563147 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.563111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:45.563332 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.563311    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:45.563409 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.563381    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:16:47.563362398 +0000 UTC m=+6.147225277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:45.664164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.664115    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:45.664355 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.664334    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:45.664459 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.664364    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:45.664459 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.664378    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:45.664459 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.664452    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:16:47.664417072 +0000 UTC m=+6.248279960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:45.875633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.875521    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 00:11:43 +0000 UTC" deadline="2028-01-16 07:12:07.768215758 +0000 UTC"
Jul 21 00:16:45.875633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.875584    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13062h55m21.892635732s"
Jul 21 00:16:45.943275 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.943245    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:45.943426 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.943360    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:45.943795 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.943766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:45.943892 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.943857    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:45.943958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:45.943927    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:45.944046 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:45.944009    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:47.477804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.477724    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:47.478212 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.477949    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:47.478212 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.478015    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:16:51.477996137 +0000 UTC m=+10.061859015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:47.579751 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.579146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:47.579751 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.579294    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:47.579751 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.579357    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:16:51.579339646 +0000 UTC m=+10.163202524 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:47.680452 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.679799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:47.680452 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.679984    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:47.680452 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.680002    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:47.680452 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.680014    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:47.680452 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.680070    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:16:51.680052618 +0000 UTC m=+10.263915510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:47.943515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.942950    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:47.943515 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.943073    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:47.943515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.943417    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:47.944072 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.943890    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:47.944072 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:47.943959    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:47.944072 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:47.944046    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:49.941282 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:49.941248    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:49.941682 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:49.941297    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:49.941682 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:49.941310    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:49.941682 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:49.941403    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:49.941682 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:49.941475    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:49.941682 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:49.941531    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:51.512505 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.512468    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:51.512943 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.512653    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:51.512943 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.512732    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:16:59.512711148 +0000 UTC m=+18.096574025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:51.613135 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.613097    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:51.613310 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.613233    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:51.613310 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.613302    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:16:59.613282799 +0000 UTC m=+18.197145688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:51.714171 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.714139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:51.714356 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.714280    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:51.714356 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.714297    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:51.714356 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.714306    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:51.714356 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.714356    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:16:59.714339993 +0000 UTC m=+18.298202867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:51.941629 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.941559    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:51.941629 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.941590    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:51.941844 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.941690    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:51.942031 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:51.941556    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:51.942143 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.942118    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:51.942233 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:51.942212    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:53.942543 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:53.942502    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:53.942989 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:53.942630    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:53.943047 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:53.942995    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:53.943095 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:53.943072    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:53.943204 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:53.943188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:53.943275 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:53.943260    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:54.996231 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:54.996037    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" event={"ID":"e952599a926de01ba0ecf526bfb18c75","Type":"ContainerStarted","Data":"e2ebf5e004e982f95111e7e6ab2beef7d0ba3896638f66c0db73849cda85d8ab"}
Jul 21 00:16:55.021003 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:55.020964    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" podStartSLOduration=13.020950394 podStartE2EDuration="13.020950394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:55.019976898 +0000 UTC m=+13.603839813" watchObservedRunningTime="2026-07-21 00:16:55.020950394 +0000 UTC m=+13.604813290"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:55.940967    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:55.941007    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:55.941086    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:55.941183    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:55.941231    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:55.941341 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:55.941312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:55.999456 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:55.999410    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m7f5t" event={"ID":"ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665","Type":"ContainerStarted","Data":"5d6b2510db158ea643b322214a310249cb5dfb6343ef4920eef4de904db4c848"}
Jul 21 00:16:56.001270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.001236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kpx88" event={"ID":"8a151b2d-b542-4a6b-99c7-5ce86b3be500","Type":"ContainerStarted","Data":"f3a11f4b25c7e94b99dbdf800ed4ba328d54f06d13ed4d1f3bcf1af0fe1edd24"}
Jul 21 00:16:56.002667 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.002642    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lqp96" event={"ID":"f3c649de-ff1c-4205-9d5e-eef443b6cc3b","Type":"ContainerStarted","Data":"ce7403c3fd832dc539282128035a896e5307c2784e07a5e3dca5a5fcbca0f3e2"}
Jul 21 00:16:56.004155 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.004123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c4924" event={"ID":"947a2b50-c63c-4e20-8e5a-04ac4ff69f0c","Type":"ContainerStarted","Data":"fda368de2a2683cdc085055fe0546971f78d8b31d7c90312de0d9cb93308137c"}
Jul 21 00:16:56.005714 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.005688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerStarted","Data":"50cf8ce33b4f3d80113b4db50adffef91bfd30a854da8b9236ec5d763909a029"}
Jul 21 00:16:56.007332 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.007312    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx" event={"ID":"f88e2920-094e-4b3d-b970-2267edefac10","Type":"ContainerStarted","Data":"8b2edeab2d7619b952a62a42679cd9bb81edcd15fbe6282eab3fd04877a1d09c"}
Jul 21 00:16:56.008768 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.008742    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"469ca48e4a79de685533b39875a7f059","Type":"ContainerStarted","Data":"17127d24ed312090a1bf65c3f5bbd5d94df19ff1f7d7b341b6dab6778a789c6c"}
Jul 21 00:16:56.013626 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.013578    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m7f5t" podStartSLOduration=3.545721951 podStartE2EDuration="14.013561839s" totalImagesPullingTime="10.467839888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:56.012905066 +0000 UTC m=+14.596767963" watchObservedRunningTime="2026-07-21 00:16:56.013561839 +0000 UTC m=+14.597424752"
Jul 21 00:16:56.032962 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.032918    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c4924" podStartSLOduration=3.35458326 podStartE2EDuration="14.032903775s" totalImagesPullingTime="10.678320515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:56.032876002 +0000 UTC m=+14.616738912" watchObservedRunningTime="2026-07-21 00:16:56.032903775 +0000 UTC m=+14.616766673"
Jul 21 00:16:56.073337 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.073298    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lqp96" podStartSLOduration=3.55731012 podStartE2EDuration="14.07328524s" totalImagesPullingTime="10.51597512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:56.072489329 +0000 UTC m=+14.656352241" watchObservedRunningTime="2026-07-21 00:16:56.07328524 +0000 UTC m=+14.657148136"
Jul 21 00:16:56.101183 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:56.101010    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kpx88" podStartSLOduration=3.59601523 podStartE2EDuration="14.100990669s" totalImagesPullingTime="10.504975439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:56.100903278 +0000 UTC m=+14.684766175" watchObservedRunningTime="2026-07-21 00:16:56.100990669 +0000 UTC m=+14.684853566"
Jul 21 00:16:57.011858 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.011819    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="50cf8ce33b4f3d80113b4db50adffef91bfd30a854da8b9236ec5d763909a029" exitCode=0
Jul 21 00:16:57.012316 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.011906    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"50cf8ce33b4f3d80113b4db50adffef91bfd30a854da8b9236ec5d763909a029"}
Jul 21 00:16:57.013554 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.013471    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67hqq" event={"ID":"783fe8c4-9e4b-42ef-82d0-7f71a93bfb0f","Type":"ContainerStarted","Data":"145173c7cc7b08c0469db7f9657e3696b8480a8b1e8ea82d514b3692555a6504"}
Jul 21 00:16:57.015342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.015318    2561 generic.go:356] "Generic (PLEG): container finished" podID="469ca48e4a79de685533b39875a7f059" containerID="17127d24ed312090a1bf65c3f5bbd5d94df19ff1f7d7b341b6dab6778a789c6c" exitCode=0
Jul 21 00:16:57.015464 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.015417    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"469ca48e4a79de685533b39875a7f059","Type":"ContainerDied","Data":"17127d24ed312090a1bf65c3f5bbd5d94df19ff1f7d7b341b6dab6778a789c6c"}
Jul 21 00:16:57.068178 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.068123    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-67hqq" podStartSLOduration=4.5360487 podStartE2EDuration="15.068102496s" totalImagesPullingTime="10.532053796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:16:57.050777185 +0000 UTC m=+15.634640083" watchObservedRunningTime="2026-07-21 00:16:57.068102496 +0000 UTC m=+15.651965394"
Jul 21 00:16:57.941579 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.941546    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:57.941781 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.941553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:57.941781 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:57.941684    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:57.941781 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:57.941564    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:57.941781 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:57.941754    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:57.941968 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:57.941836    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:16:58.297840 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:58.297789    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:58.298414 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:58.298375    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:16:59.576365 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.576327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:59.576879 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.576506    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:59.576879 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.576580    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:17:15.576564557 +0000 UTC m=+34.160427431 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:16:59.677037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.676999    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:59.677201 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.677133    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:59.677201 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.677195    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:15.67717835 +0000 UTC m=+34.261041225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:16:59.778341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.778305    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:59.778542 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.778519    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:16:59.778600 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.778553    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:16:59.778600 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.778566    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:59.778671 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.778628    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:17:15.778609685 +0000 UTC m=+34.362472578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:16:59.940889 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.940817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:16:59.941019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.940817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:16:59.941019 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.940933    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:16:59.941019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:16:59.940828    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:16:59.941019 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.940995    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:16:59.941153 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:16:59.941090    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:01.941928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:01.941894    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:01.942538 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:01.942014    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:01.942602 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:01.942558    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:01.942662 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:01.942646    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:01.942713 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:01.942693    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:01.942761 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:01.942743    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:03.940747 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:03.940708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:03.941470 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:03.940708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:03.941470 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:03.940844    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:03.941470 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:03.940852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:03.941470 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:03.940918    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:03.941470 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:03.941039    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:04.651312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:04.651169    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 00:17:04.902970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:04.902622    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T00:17:04.651309572Z","UUID":"799bae82-9b31-43d2-89d2-e7de47b5291d","Handler":null,"Name":"","Endpoint":""}
Jul 21 00:17:04.904089 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:04.904068    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 00:17:04.904171 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:04.904112    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 00:17:05.032109 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.032078    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vzcbb" event={"ID":"81869e75-484e-4145-b3d0-1a9261042b4f","Type":"ContainerStarted","Data":"0a1e036af7ca6ffe6a97413d1696b89b7c2683d7f7aff6046662f1aacbd8acd3"}
Jul 21 00:17:05.034496 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.034420    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx" event={"ID":"f88e2920-094e-4b3d-b970-2267edefac10","Type":"ContainerStarted","Data":"60b9022237893e2095393bc3ac7044a02d9b230145f0ccbb2d5c91a9c60adb32"}
Jul 21 00:17:05.038717 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038689    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"bb64d816004bcbf8f64385c72bf24f69965a5625b7cd1e98e7975360c5af3fa2"}
Jul 21 00:17:05.038844 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"98818980322f07dfbf4371dfba096444852b7112c3abafd82293f35aedbff6d8"}
Jul 21 00:17:05.038844 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038827    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"9e8a5660fd293091929951b3b4b9689c792d0894b22e2c9e7a3d27c60be6a2b8"}
Jul 21 00:17:05.038953 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038877    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"17e15489546ecce6dd68e9d7c9ab7357f41d5d4ed6d79999703858ad0dfede6b"}
Jul 21 00:17:05.038953 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038923    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"b3f0dfe93f6a92c30474bb3e28ff74ae6d2787186e722204691e05cef046d8c5"}
Jul 21 00:17:05.039037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.038969    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"ad5f16d3adac55df4e54ba6c3fbe8b05f6fa58e47f88592f237cd6a4095b306f"}
Jul 21 00:17:05.042967 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.042940    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"469ca48e4a79de685533b39875a7f059","Type":"ContainerStarted","Data":"841389c9282fc375f63d0f7214393dc31cbf7279766f2b762100d63ee32fdac8"}
Jul 21 00:17:05.052183 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.052132    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vzcbb" podStartSLOduration=2.766803926 podStartE2EDuration="23.052119394s" totalImagesPullingTime="20.285315468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:05.051962918 +0000 UTC m=+23.635825814" watchObservedRunningTime="2026-07-21 00:17:05.052119394 +0000 UTC m=+23.635982291"
Jul 21 00:17:05.067704 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.067656    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" podStartSLOduration=23.067637042 podStartE2EDuration="23.067637042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:05.066972178 +0000 UTC m=+23.650835076" watchObservedRunningTime="2026-07-21 00:17:05.067637042 +0000 UTC m=+23.651499945"
Jul 21 00:17:05.941279 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.941239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:05.941492 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:05.941362    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:05.941492 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.941239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:05.941492 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:05.941239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:05.941636 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:05.941488    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:05.941636 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:05.941550    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:06.046465 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:06.046412    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx" event={"ID":"f88e2920-094e-4b3d-b970-2267edefac10","Type":"ContainerStarted","Data":"5c45c512d09438d3fe652f65fded1d79f004dc04ddcbfd85676657528a4adfbc"}
Jul 21 00:17:06.065309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:06.065258    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5hlx" podStartSLOduration=3.772718686 podStartE2EDuration="24.065243489s" totalImagesPullingTime="20.292524803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:06.06481312 +0000 UTC m=+24.648676027" watchObservedRunningTime="2026-07-21 00:17:06.065243489 +0000 UTC m=+24.649106401"
Jul 21 00:17:06.283954 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:06.283918    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:17:06.284204 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:06.284079    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 00:17:06.284684 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:06.284661    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lqp96"
Jul 21 00:17:07.940814 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:07.940777    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:07.941314 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:07.940777    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:07.941314 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:07.940907    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:07.941314 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:07.940992    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:07.941314 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:07.940789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:07.941314 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:07.941157    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:08.054424 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:08.054391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"824438621ef49f0d5cbdfb9cc09c066d56f73ca859062c5fe41cdcbac10a8a6b"}
Jul 21 00:17:09.945312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:09.944534    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:09.945312 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:09.944757    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:09.945312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:09.945126    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:09.945312 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:09.945213    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:09.945312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:09.945259    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:09.946093 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:09.945339    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:10.063012 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:10.062735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" event={"ID":"2d686cd2-f11a-4e5c-b805-2620d608e3fc","Type":"ContainerStarted","Data":"5d44fc1e4a4274f3fdc3b6dee79ff4add71de090e426a8f731ae087595de9b5e"}
Jul 21 00:17:10.063170 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:10.063060    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:10.064927 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:10.064898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerStarted","Data":"1e9d4ffdb17b790f6ceddcd5c3045185d53f94cdad626d8c3bba1a3c15c053c7"}
Jul 21 00:17:10.082502 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:10.082474    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:10.088725 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:10.088680    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" podStartSLOduration=7.75260633 podStartE2EDuration="28.088668791s" totalImagesPullingTime="20.336062461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:10.088502026 +0000 UTC m=+28.672364944" watchObservedRunningTime="2026-07-21 00:17:10.088668791 +0000 UTC m=+28.672531687"
Jul 21 00:17:11.068390 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.068357    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="1e9d4ffdb17b790f6ceddcd5c3045185d53f94cdad626d8c3bba1a3c15c053c7" exitCode=0
Jul 21 00:17:11.068853 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.068463    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"1e9d4ffdb17b790f6ceddcd5c3045185d53f94cdad626d8c3bba1a3c15c053c7"}
Jul 21 00:17:11.068853 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.068516    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 00:17:11.068932 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.068918    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:11.086550 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.086532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:11.781162 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.781122    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6z2j"]
Jul 21 00:17:11.781325 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.781256    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:11.781383 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:11.781354    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:11.784427 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.784396    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-987l2"]
Jul 21 00:17:11.784582 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.784547    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:11.784683 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:11.784661    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:11.785027 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.785008    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j4jz9"]
Jul 21 00:17:11.785120 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:11.785096    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:11.785206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:11.785189    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:12.071101 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:12.071027    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 00:17:13.073136 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:13.073105    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 00:17:13.941510 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:13.941484    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:13.941510 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:13.941507    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:13.941726 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:13.941490    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:13.941726 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:13.941617    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:13.941827 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:13.941775    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:13.941896 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:13.941868    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:14.077564 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.077480    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="65dc12f29a8b69a80e52c020445fd0ad7dd5ae9590f72a73e4a7f59222549cdd" exitCode=0
Jul 21 00:17:14.077564 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.077529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"65dc12f29a8b69a80e52c020445fd0ad7dd5ae9590f72a73e4a7f59222549cdd"}
Jul 21 00:17:14.100122 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.100099    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:14.100309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.100297    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 00:17:14.113144 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.113114    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" podUID="2d686cd2-f11a-4e5c-b805-2620d608e3fc" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 21 00:17:14.122482 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:14.122413    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc" podUID="2d686cd2-f11a-4e5c-b805-2620d608e3fc" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 21 00:17:15.601343 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.601161    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:15.601730 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.601311    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 00:17:15.601730 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.601416    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret podName:3b9d1a2f-bd64-446d-a70d-fbff3a78b59f nodeName:}" failed. No retries permitted until 2026-07-21 00:17:47.601400439 +0000 UTC m=+66.185263315 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret") pod "global-pull-secret-syncer-t6z2j" (UID: "3b9d1a2f-bd64-446d-a70d-fbff3a78b59f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 00:17:15.702331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.702257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:15.702484 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.702359    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:17:15.702484 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.702415    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:47.702401938 +0000 UTC m=+66.286264813 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 00:17:15.802884 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.802856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:15.803041 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.803016    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 00:17:15.803041 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.803036    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 00:17:15.803106 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.803045    2561 projected.go:196] Error preparing data for projected volume kube-api-access-vgm8d for pod openshift-network-diagnostics/network-check-target-j4jz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:17:15.803106 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.803091    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d podName:1d3f4310-a086-4afe-b450-9d8fcac625ef nodeName:}" failed. No retries permitted until 2026-07-21 00:17:47.80307817 +0000 UTC m=+66.386941045 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vgm8d" (UniqueName: "kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d") pod "network-check-target-j4jz9" (UID: "1d3f4310-a086-4afe-b450-9d8fcac625ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 00:17:15.940830 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.940803    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:15.940931 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.940804    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:15.940975 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.940936    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4jz9" podUID="1d3f4310-a086-4afe-b450-9d8fcac625ef"
Jul 21 00:17:15.941021 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:15.940803    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:15.941059 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.941018    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6z2j" podUID="3b9d1a2f-bd64-446d-a70d-fbff3a78b59f"
Jul 21 00:17:15.941128 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:15.941108    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:17:16.084474 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:16.084426    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="ee6a15665327933d8c3e13e4fcbd705bc541fe136ec5f8aae48d3c4f636704eb" exitCode=0
Jul 21 00:17:16.084627 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:16.084515    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"ee6a15665327933d8c3e13e4fcbd705bc541fe136ec5f8aae48d3c4f636704eb"}
Jul 21 00:17:17.277361 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.277331    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeReady"
Jul 21 00:17:17.277919 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.277508    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 00:17:17.314295 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.314265    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"]
Jul 21 00:17:17.316505 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.316481    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.318181 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.318159    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:17:17.319058 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.319037    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 21 00:17:17.319257 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.319242    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 21 00:17:17.319530 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.319477    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7r9b4\""
Jul 21 00:17:17.319853 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.319836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.323573 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.323275    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 00:17:17.324857 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.324835    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 00:17:17.325044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.325024    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 00:17:17.325417 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.325401    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hclsz\""
Jul 21 00:17:17.328718 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.328699    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 00:17:17.329867 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.329845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"]
Jul 21 00:17:17.334483 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.333695    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q9ctc"]
Jul 21 00:17:17.335821 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.335803    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c778b"]
Jul 21 00:17:17.335980 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.335944    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.337388 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.337370    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.339274 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.339258    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26lqt\""
Jul 21 00:17:17.339274 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.339272    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 00:17:17.340013 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.339981    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 00:17:17.340104 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.339986    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 00:17:17.340104 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.340086    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:17:17.341141 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.340610    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 00:17:17.341141 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.340829    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 00:17:17.341141 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.341004    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-znvqg\""
Jul 21 00:17:17.351951 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.351929    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c778b"]
Jul 21 00:17:17.352403 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.352385    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9ctc"]
Jul 21 00:17:17.415794 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415767    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.415794 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415800    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415817    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k57fh\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415842    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.416034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415940    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.415978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.416016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.416215 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.416077    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416215 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.416141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.416215 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.416175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.516687 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516648    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-cert\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.516863 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516713    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3a03790-c202-4c90-9964-42b29b65a1d6-config-volume\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.516863 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.516863 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516841    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.517022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.517022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.517022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k57fh\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.517022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516966    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.517022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.516991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.517453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.517394    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.517617    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.517621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.517646    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.517670    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a03790-c202-4c90-9964-42b29b65a1d6-tmp-dir\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.517852    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv4cb\" (UniqueName: \"kubernetes.io/projected/c3a03790-c202-4c90-9964-42b29b65a1d6-kube-api-access-vv4cb\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.518001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.518045    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7q6c\" (UniqueName: \"kubernetes.io/projected/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-kube-api-access-g7q6c\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.518201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.518308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.518579    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:18.018500938 +0000 UTC m=+36.602363813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.519162    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:18.019056226 +0000 UTC m=+36.602919105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:17.520767 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.520375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.522560 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.522532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:17.522750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.522729    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.523237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.523220    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.523305 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.523247    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.527164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.527144    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k57fh\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.527585 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.527542    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:17.620146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a03790-c202-4c90-9964-42b29b65a1d6-tmp-dir\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.620146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv4cb\" (UniqueName: \"kubernetes.io/projected/c3a03790-c202-4c90-9964-42b29b65a1d6-kube-api-access-vv4cb\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.620373 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620168    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7q6c\" (UniqueName: \"kubernetes.io/projected/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-kube-api-access-g7q6c\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.620373 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-cert\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.620373 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3a03790-c202-4c90-9964-42b29b65a1d6-config-volume\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.620557 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620416    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.620557 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620496    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a03790-c202-4c90-9964-42b29b65a1d6-tmp-dir\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.620647 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.620562    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:17.620647 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:17.620625    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:18.120606046 +0000 UTC m=+36.704468935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:17.620766 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.620749    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3a03790-c202-4c90-9964-42b29b65a1d6-config-volume\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.622943 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.622914    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-cert\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.629102 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.629084    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv4cb\" (UniqueName: \"kubernetes.io/projected/c3a03790-c202-4c90-9964-42b29b65a1d6-kube-api-access-vv4cb\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:17.630337 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.630316    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7q6c\" (UniqueName: \"kubernetes.io/projected/51ffe5ee-1c35-4eb8-a409-f3281f4b6797-kube-api-access-g7q6c\") pod \"ingress-canary-c778b\" (UID: \"51ffe5ee-1c35-4eb8-a409-f3281f4b6797\") " pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.652325 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.652300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c778b"
Jul 21 00:17:17.787556 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.787464    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c778b"]
Jul 21 00:17:17.792961 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:17:17.792776    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ffe5ee_1c35_4eb8_a409_f3281f4b6797.slice/crio-e5cf576c7630c264524a933bfc1126cd6cb0bb1f387a2753e2fbb9340d856ea6 WatchSource:0}: Error finding container e5cf576c7630c264524a933bfc1126cd6cb0bb1f387a2753e2fbb9340d856ea6: Status 404 returned error can't find the container with id e5cf576c7630c264524a933bfc1126cd6cb0bb1f387a2753e2fbb9340d856ea6
Jul 21 00:17:17.941279 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.941247    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:17.941279 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.941261    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:17.941562 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.941476    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:17.944150 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944125    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 00:17:17.944252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944125    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m4mrq\""
Jul 21 00:17:17.944471 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944449    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 00:17:17.944601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944492    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 00:17:17.944601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944492    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 00:17:17.944601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:17.944492    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztz7f\""
Jul 21 00:17:18.023766 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:18.023731    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:18.023887 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:18.023781    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:18.023947 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.023881    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:18.023947 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.023900    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:18.023947 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.023920    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:18.024061 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.023974    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:19.023954524 +0000 UTC m=+37.607817420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:18.024061 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.023988    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:19.023981635 +0000 UTC m=+37.607844511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:18.090939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:18.090816    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c778b" event={"ID":"51ffe5ee-1c35-4eb8-a409-f3281f4b6797","Type":"ContainerStarted","Data":"e5cf576c7630c264524a933bfc1126cd6cb0bb1f387a2753e2fbb9340d856ea6"}
Jul 21 00:17:18.124312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:18.124281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:18.124486 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.124467    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:18.124553 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:18.124532    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:19.124517119 +0000 UTC m=+37.708379996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:19.032804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:19.032767    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:19.032812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.032930    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.032948    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.033012    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:21.032996286 +0000 UTC m=+39.616859162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.033021    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:19.033372 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.033091    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:21.033073971 +0000 UTC m=+39.616936848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:19.133869 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:19.133828    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:19.134037 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.133976    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:19.134116 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:19.134044    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:21.134024879 +0000 UTC m=+39.717887756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:21.049935 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:21.049903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:21.049935 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:21.049940    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:21.050321 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.050039    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:21.050321 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.050047    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:21.050321 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.050065    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:21.050321 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.050094    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:25.050078591 +0000 UTC m=+43.633941466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:21.050321 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.050113    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:25.050100882 +0000 UTC m=+43.633963757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:21.150639 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:21.150602    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:21.150812 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.150761    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:21.150879 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:21.150836    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:25.150816195 +0000 UTC m=+43.734679070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:23.105848 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:23.105813    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="b140d3af825fae48200186219c80e8640bf51a376500bf151b9d1eda6d871ab5" exitCode=0
Jul 21 00:17:23.106266 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:23.105898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"b140d3af825fae48200186219c80e8640bf51a376500bf151b9d1eda6d871ab5"}
Jul 21 00:17:23.107341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:23.107321    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c778b" event={"ID":"51ffe5ee-1c35-4eb8-a409-f3281f4b6797","Type":"ContainerStarted","Data":"bbeb876c5185a70aa8b3212fb60ed57fd774af63c2eaad0de77dbe3923dd585e"}
Jul 21 00:17:23.152157 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:23.152095    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c778b" podStartSLOduration=1.707955719 podStartE2EDuration="6.152076024s" totalImagesPullingTime="4.444120305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:23.151658166 +0000 UTC m=+41.735521062" watchObservedRunningTime="2026-07-21 00:17:23.152076024 +0000 UTC m=+41.735938923"
Jul 21 00:17:24.113169 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:24.112984    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a312e9c-35ec-4ea9-ac0e-f25cf30d3938" containerID="0ca378c2bdc3ee1222437eb5a37d78571af6613991906b584feae4bc1dd84289" exitCode=0
Jul 21 00:17:24.113560 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:24.113059    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerDied","Data":"0ca378c2bdc3ee1222437eb5a37d78571af6613991906b584feae4bc1dd84289"}
Jul 21 00:17:25.078807 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:25.078765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:25.078807 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:25.078812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:25.079039 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.078927    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:25.079039 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.078941    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:25.079039 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.078964    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:25.079039 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.078994    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:33.078977663 +0000 UTC m=+51.662840557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:25.079039 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.079020    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:33.079000679 +0000 UTC m=+51.662863580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:25.119570 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:25.119532    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" event={"ID":"3a312e9c-35ec-4ea9-ac0e-f25cf30d3938","Type":"ContainerStarted","Data":"c486704b57f44cd35cd2bd582f05ef1143be767087f6892ba3b82f1d41c04e06"}
Jul 21 00:17:25.143110 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:25.143063    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hhlpr" podStartSLOduration=10.604500359 podStartE2EDuration="43.143048533s" totalImagesPullingTime="32.538548174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:25.141818072 +0000 UTC m=+43.725680990" watchObservedRunningTime="2026-07-21 00:17:25.143048533 +0000 UTC m=+43.726911428"
Jul 21 00:17:25.179771 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:25.179739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:25.179916 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.179880    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:25.179982 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:25.179941    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:33.179922594 +0000 UTC m=+51.763785474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:33.137885 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:33.137852    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:33.137893    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.138006    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.138010    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.138026    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.138074    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:49.1380538 +0000 UTC m=+67.721916676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:33.138503 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.138091    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:17:49.138080117 +0000 UTC m=+67.721943014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:33.238664 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:33.238631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:33.238836 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.238783    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:33.238889 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:33.238841    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:17:49.238825651 +0000 UTC m=+67.822688526 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:44.123157 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:44.123129    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qdxc"
Jul 21 00:17:47.640930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.640894    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:47.643722 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.643705    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 00:17:47.655917 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.655888    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b9d1a2f-bd64-446d-a70d-fbff3a78b59f-original-pull-secret\") pod \"global-pull-secret-syncer-t6z2j\" (UID: \"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f\") " pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:47.741504 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.741471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:17:47.744417 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.744396    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 00:17:47.752621 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:47.752605    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 21 00:17:47.752700 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:47.752658    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:18:51.752642468 +0000 UTC m=+130.336505342 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : secret "metrics-daemon-secret" not found
Jul 21 00:17:47.842299 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.842270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:47.844913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.844894    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 00:17:47.855420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.855397    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 00:17:47.865682 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.865657    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgm8d\" (UniqueName: \"kubernetes.io/projected/1d3f4310-a086-4afe-b450-9d8fcac625ef-kube-api-access-vgm8d\") pod \"network-check-target-j4jz9\" (UID: \"1d3f4310-a086-4afe-b450-9d8fcac625ef\") " pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:47.953283 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.953228    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6z2j"
Jul 21 00:17:47.967348 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.967329    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m4mrq\""
Jul 21 00:17:47.975864 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:47.975845    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:48.096611 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:48.096570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6z2j"]
Jul 21 00:17:48.099596 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:17:48.099570    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9d1a2f_bd64_446d_a70d_fbff3a78b59f.slice/crio-441ca18630105d9b52d441cfd82e201d4baaf3b5b22e7647451e49fa4c547b4d WatchSource:0}: Error finding container 441ca18630105d9b52d441cfd82e201d4baaf3b5b22e7647451e49fa4c547b4d: Status 404 returned error can't find the container with id 441ca18630105d9b52d441cfd82e201d4baaf3b5b22e7647451e49fa4c547b4d
Jul 21 00:17:48.121676 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:48.121655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j4jz9"]
Jul 21 00:17:48.124411 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:17:48.124390    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3f4310_a086_4afe_b450_9d8fcac625ef.slice/crio-bae056c4347eba9a20d8e0c6261b303085e636547bc41b9517f907852a9a049a WatchSource:0}: Error finding container bae056c4347eba9a20d8e0c6261b303085e636547bc41b9517f907852a9a049a: Status 404 returned error can't find the container with id bae056c4347eba9a20d8e0c6261b303085e636547bc41b9517f907852a9a049a
Jul 21 00:17:48.175045 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:48.175018    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j4jz9" event={"ID":"1d3f4310-a086-4afe-b450-9d8fcac625ef","Type":"ContainerStarted","Data":"bae056c4347eba9a20d8e0c6261b303085e636547bc41b9517f907852a9a049a"}
Jul 21 00:17:48.175983 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:48.175959    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6z2j" event={"ID":"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f","Type":"ContainerStarted","Data":"441ca18630105d9b52d441cfd82e201d4baaf3b5b22e7647451e49fa4c547b4d"}
Jul 21 00:17:49.153028 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:49.152988    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:49.153037    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.153132    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.153143    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.153162    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.153207    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:18:21.153193587 +0000 UTC m=+99.737056508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:17:49.153508 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.153225    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:18:21.153213647 +0000 UTC m=+99.737076521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:17:49.253741 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:49.253707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:17:49.253946 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.253900    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:17:49.254004 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:17:49.253963    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:18:21.253944839 +0000 UTC m=+99.837807714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:17:53.190555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:53.190472    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j4jz9" event={"ID":"1d3f4310-a086-4afe-b450-9d8fcac625ef","Type":"ContainerStarted","Data":"b4010fec89e3f04fb865496afa517c9df3751640537aafb61689624686efcabc"}
Jul 21 00:17:53.190981 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:53.190596    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:17:53.191868 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:53.191848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6z2j" event={"ID":"3b9d1a2f-bd64-446d-a70d-fbff3a78b59f","Type":"ContainerStarted","Data":"28b885acc9e87dcb7533b873f7a8f4da2c92831e8960dfbf8b3e37d84943e12a"}
Jul 21 00:17:53.207573 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:53.207531    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j4jz9" podStartSLOduration=66.499996705 podStartE2EDuration="1m11.207517748s" totalImagesPullingTime="4.707521043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:53.207147292 +0000 UTC m=+71.791010188" watchObservedRunningTime="2026-07-21 00:17:53.207517748 +0000 UTC m=+71.791380644"
Jul 21 00:17:53.224698 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:17:53.224661    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t6z2j" podStartSLOduration=66.488709634 podStartE2EDuration="1m11.224649367s" totalImagesPullingTime="4.735939733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:17:53.224487641 +0000 UTC m=+71.808350546" watchObservedRunningTime="2026-07-21 00:17:53.224649367 +0000 UTC m=+71.808512263"
Jul 21 00:18:21.189683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:18:21.189534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:18:21.189683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:18:21.189604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:18:21.190206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.189702    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:18:21.190206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.189719    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:18:21.190206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.189720    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:18:21.190206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.189799    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:19:25.18978396 +0000 UTC m=+163.773646836 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:18:21.190206 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.189817    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:19:25.18980904 +0000 UTC m=+163.773671915 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:18:21.290477 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:18:21.290405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:18:21.290608 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.290559    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:18:21.290649 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:21.290633    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:19:25.290617689 +0000 UTC m=+163.874480563 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:18:24.196535 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:18:24.196507    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j4jz9"
Jul 21 00:18:51.808650 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:18:51.808593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:18:51.809175 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:51.808741    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 21 00:18:51.809175 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:18:51.808811    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:53.808794069 +0000 UTC m=+252.392656946 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : secret "metrics-daemon-secret" not found
Jul 21 00:19:20.330242 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:20.330186    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs" podUID="46bee106-bf85-4b5f-90a1-ef0cfafdcc86"
Jul 21 00:19:20.337319 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:20.337295    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d64586967-rvs5d" podUID="e987206e-2e14-4244-aee3-a4e8466d508b"
Jul 21 00:19:20.347492 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:20.347460    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9ctc" podUID="c3a03790-c202-4c90-9964-42b29b65a1d6"
Jul 21 00:19:20.394562 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:20.394538    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:19:20.394562 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:20.394549    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:19:20.394726 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:20.394538    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:19:20.961490 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:20.961420    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:19:25.235919 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:25.235873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:25.235980    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.236021    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.236069    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.236083    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d64586967-rvs5d: secret "image-registry-tls" not found
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.236093    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert podName:46bee106-bf85-4b5f-90a1-ef0cfafdcc86 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:27.236077362 +0000 UTC m=+285.819940237 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8ptrs" (UID: "46bee106-bf85-4b5f-90a1-ef0cfafdcc86") : secret "networking-console-plugin-cert" not found
Jul 21 00:19:25.236334 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.236118    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls podName:e987206e-2e14-4244-aee3-a4e8466d508b nodeName:}" failed. No retries permitted until 2026-07-21 00:21:27.236107577 +0000 UTC m=+285.819970453 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls") pod "image-registry-d64586967-rvs5d" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b") : secret "image-registry-tls" not found
Jul 21 00:19:25.336515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:25.336463    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:19:25.336724 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.336632    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 00:19:25.336724 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:19:25.336714    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls podName:c3a03790-c202-4c90-9964-42b29b65a1d6 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:27.336693739 +0000 UTC m=+285.920556615 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls") pod "dns-default-q9ctc" (UID: "c3a03790-c202-4c90-9964-42b29b65a1d6") : secret "dns-default-metrics-tls" not found
Jul 21 00:19:34.941577 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:19:34.941493    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:20:53.838052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:53.838006    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:20:53.838559 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:53.838147    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 21 00:20:53.838559 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:53.838219    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs podName:d9d8c49a-18f0-46ae-af48-e8dcb11c3a55 nodeName:}" failed. No retries permitted until 2026-07-21 00:22:55.83820454 +0000 UTC m=+374.422067414 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs") pod "network-metrics-daemon-987l2" (UID: "d9d8c49a-18f0-46ae-af48-e8dcb11c3a55") : secret "metrics-daemon-secret" not found
Jul 21 00:20:54.613284 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.613257    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"]
Jul 21 00:20:54.615969 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.615945    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"
Jul 21 00:20:54.620890 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.620861    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"]
Jul 21 00:20:54.621742 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.621713    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.621869 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.621742    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tgntf\""
Jul 21 00:20:54.621869 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.621775    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.625011 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.624994    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"]
Jul 21 00:20:54.625112 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.625098    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.627703 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.627684    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"]
Jul 21 00:20:54.627803 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.627786    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.627868 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.627801    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5sp8q\""
Jul 21 00:20:54.628353 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.628338    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.628456 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.628422    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 21 00:20:54.628511 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.628432    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.628511 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.628430    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 21 00:20:54.630572 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.630557    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.633637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.633622    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.633973 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.633953    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 21 00:20:54.634039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.634012    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 21 00:20:54.634100 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.634019    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.635298 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.635281    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wktn2\""
Jul 21 00:20:54.635375 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.635315    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.635375 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.635329    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.635486 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.635377    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sdkjf\""
Jul 21 00:20:54.635708 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.635693    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 21 00:20:54.641903 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.641883    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"]
Jul 21 00:20:54.643052 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.643036    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"]
Jul 21 00:20:54.644131 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"]
Jul 21 00:20:54.644555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644398    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfnk5\" (UniqueName: \"kubernetes.io/projected/df455d46-57db-4f46-b9ab-75f218dd5aee-kube-api-access-mfnk5\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.644555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df455d46-57db-4f46-b9ab-75f218dd5aee-config\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.644555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lccn\" (UniqueName: \"kubernetes.io/projected/1b7c55fb-7095-405c-b7a9-bbbbef6d203c-kube-api-access-7lccn\") pod \"volume-data-source-validator-78cf4c6f6-q7hck\" (UID: \"1b7c55fb-7095-405c-b7a9-bbbbef6d203c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"
Jul 21 00:20:54.644555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644519    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.644820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.644820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.644820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644686    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj22f\" (UniqueName: \"kubernetes.io/projected/a21bfe0c-4964-4d72-a133-0c4e8429aae7-kube-api-access-mj22f\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.644820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644776    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df455d46-57db-4f46-b9ab-75f218dd5aee-serving-cert\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.645009 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.644832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24cj\" (UniqueName: \"kubernetes.io/projected/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-kube-api-access-v24cj\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.647631 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.647610    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"]
Jul 21 00:20:54.708776 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.708757    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"]
Jul 21 00:20:54.711543 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.711519    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.712731 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.712713    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-f8n4k"]
Jul 21 00:20:54.713750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.713733    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 21 00:20:54.714009 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.713992    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.714062 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.714029    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.714062 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.714050    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 21 00:20:54.714514 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.714501    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qr46m\""
Jul 21 00:20:54.715204 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.715189    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-vqlfm"]
Jul 21 00:20:54.715331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.715316    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.718012 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.717994    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.718683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.718662    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 21 00:20:54.718783 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.718694    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.718783 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.718772    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wfdzk\""
Jul 21 00:20:54.718987 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.718971    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.719837 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.719820    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 21 00:20:54.722233 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.722214    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 00:20:54.723459 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.723428    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 21 00:20:54.723571 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.723460    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kxnpb\""
Jul 21 00:20:54.723571 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.723460    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 21 00:20:54.725371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.725350    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"]
Jul 21 00:20:54.726866 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.726849    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 21 00:20:54.727224 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.727205    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 00:20:54.729668 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.729621    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 21 00:20:54.735603 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.735090    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-f8n4k"]
Jul 21 00:20:54.736222 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.736199    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-vqlfm"]
Jul 21 00:20:54.745636 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745612    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-service-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.745737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745642    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.745737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.745737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745679    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hvz7\" (UniqueName: \"kubernetes.io/projected/18c75a0d-5efe-450c-baff-342084431ad8-kube-api-access-7hvz7\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.745737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745699    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-serving-cert\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.745949 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:54.745794    2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 00:20:54.745949 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj22f\" (UniqueName: \"kubernetes.io/projected/a21bfe0c-4964-4d72-a133-0c4e8429aae7-kube-api-access-mj22f\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.745949 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:54.745877    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls podName:a21bfe0c-4964-4d72-a133-0c4e8429aae7 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:55.245861686 +0000 UTC m=+253.829724561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zkxx6" (UID: "a21bfe0c-4964-4d72-a133-0c4e8429aae7") : secret "samples-operator-tls" not found
Jul 21 00:20:54.746106 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.745999    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df455d46-57db-4f46-b9ab-75f218dd5aee-serving-cert\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.746106 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0052f0d2-f154-44a7-aea7-40e1086be706-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.746106 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746052    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-config\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.746106 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746102    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df455d46-57db-4f46-b9ab-75f218dd5aee-config\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.746302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746155    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0052f0d2-f154-44a7-aea7-40e1086be706-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.746302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-trusted-ca\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.746302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746213    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24cj\" (UniqueName: \"kubernetes.io/projected/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-kube-api-access-v24cj\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.746302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746259    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lccn\" (UniqueName: \"kubernetes.io/projected/1b7c55fb-7095-405c-b7a9-bbbbef6d203c-kube-api-access-7lccn\") pod \"volume-data-source-validator-78cf4c6f6-q7hck\" (UID: \"1b7c55fb-7095-405c-b7a9-bbbbef6d203c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"
Jul 21 00:20:54.746302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746295    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-tmp\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:54.746415    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746453    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18c75a0d-5efe-450c-baff-342084431ad8-serving-cert\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:54.746496    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:55.246479125 +0000 UTC m=+253.830342004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746518    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7mp\" (UniqueName: \"kubernetes.io/projected/0052f0d2-f154-44a7-aea7-40e1086be706-kube-api-access-lz7mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.746567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mckcs\" (UniqueName: \"kubernetes.io/projected/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-kube-api-access-mckcs\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.746828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746599    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-snapshots\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.746828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746621    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.746828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746641    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df455d46-57db-4f46-b9ab-75f218dd5aee-config\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.746828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746669    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfnk5\" (UniqueName: \"kubernetes.io/projected/df455d46-57db-4f46-b9ab-75f218dd5aee-kube-api-access-mfnk5\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.746828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.746699    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.748404 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.748389    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df455d46-57db-4f46-b9ab-75f218dd5aee-serving-cert\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.760486 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.760466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj22f\" (UniqueName: \"kubernetes.io/projected/a21bfe0c-4964-4d72-a133-0c4e8429aae7-kube-api-access-mj22f\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:54.760592 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.760535    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfnk5\" (UniqueName: \"kubernetes.io/projected/df455d46-57db-4f46-b9ab-75f218dd5aee-kube-api-access-mfnk5\") pod \"service-ca-operator-5bcbc94c78-f4gc7\" (UID: \"df455d46-57db-4f46-b9ab-75f218dd5aee\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:54.760592 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.760560    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24cj\" (UniqueName: \"kubernetes.io/projected/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-kube-api-access-v24cj\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:54.760723 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.760690    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lccn\" (UniqueName: \"kubernetes.io/projected/1b7c55fb-7095-405c-b7a9-bbbbef6d203c-kube-api-access-7lccn\") pod \"volume-data-source-validator-78cf4c6f6-q7hck\" (UID: \"1b7c55fb-7095-405c-b7a9-bbbbef6d203c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"
Jul 21 00:20:54.847532 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-snapshots\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847539    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-service-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847658    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hvz7\" (UniqueName: \"kubernetes.io/projected/18c75a0d-5efe-450c-baff-342084431ad8-kube-api-access-7hvz7\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-serving-cert\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0052f0d2-f154-44a7-aea7-40e1086be706-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-config\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847777    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0052f0d2-f154-44a7-aea7-40e1086be706-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847803    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-trusted-ca\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-tmp\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847871    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18c75a0d-5efe-450c-baff-342084431ad8-serving-cert\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847898    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7mp\" (UniqueName: \"kubernetes.io/projected/0052f0d2-f154-44a7-aea7-40e1086be706-kube-api-access-lz7mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.847941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.847947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mckcs\" (UniqueName: \"kubernetes.io/projected/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-kube-api-access-mckcs\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.848553 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.848265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-service-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.848553 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.848296    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0052f0d2-f154-44a7-aea7-40e1086be706-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.848553 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.848527    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c75a0d-5efe-450c-baff-342084431ad8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.848695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.848608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-tmp\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.848783 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.848750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-config\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.849160 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.849135    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18c75a0d-5efe-450c-baff-342084431ad8-snapshots\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.849302 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.849284    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-trusted-ca\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.850266 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.850243    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-serving-cert\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.850266 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.850257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18c75a0d-5efe-450c-baff-342084431ad8-serving-cert\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.850499 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.850483    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0052f0d2-f154-44a7-aea7-40e1086be706-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.856305 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.856283    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hvz7\" (UniqueName: \"kubernetes.io/projected/18c75a0d-5efe-450c-baff-342084431ad8-kube-api-access-7hvz7\") pod \"insights-operator-555d9d7885-vqlfm\" (UID: \"18c75a0d-5efe-450c-baff-342084431ad8\") " pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:54.856419 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.856355    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7mp\" (UniqueName: \"kubernetes.io/projected/0052f0d2-f154-44a7-aea7-40e1086be706-kube-api-access-lz7mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-sjjsp\" (UID: \"0052f0d2-f154-44a7-aea7-40e1086be706\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:54.856499 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.856428    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mckcs\" (UniqueName: \"kubernetes.io/projected/4e48fe7f-30e5-402f-b4f5-ceaf629ca176-kube-api-access-mckcs\") pod \"console-operator-565b7b569d-f8n4k\" (UID: \"4e48fe7f-30e5-402f-b4f5-ceaf629ca176\") " pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:54.930470 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.930394    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"
Jul 21 00:20:54.935673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:54.935654    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"
Jul 21 00:20:55.021726 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.021706    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"
Jul 21 00:20:55.028829 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.028802    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:20:55.038091 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.037921    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-vqlfm"
Jul 21 00:20:55.069852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.069522    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck"]
Jul 21 00:20:55.073407 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:20:55.073376    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7c55fb_7095_405c_b7a9_bbbbef6d203c.slice/crio-798d5e7db0d398c0a93facacd9c3de3680144b9e937b76f57f4be9e652dcf90f WatchSource:0}: Error finding container 798d5e7db0d398c0a93facacd9c3de3680144b9e937b76f57f4be9e652dcf90f: Status 404 returned error can't find the container with id 798d5e7db0d398c0a93facacd9c3de3680144b9e937b76f57f4be9e652dcf90f
Jul 21 00:20:55.086494 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.086420    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7"]
Jul 21 00:20:55.171556 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:20:55.171518    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0052f0d2_f154_44a7_aea7_40e1086be706.slice/crio-b6830d9f8f5410fa596f6457006c9ccbb2130916df6d34e4ef282c887843a1cc WatchSource:0}: Error finding container b6830d9f8f5410fa596f6457006c9ccbb2130916df6d34e4ef282c887843a1cc: Status 404 returned error can't find the container with id b6830d9f8f5410fa596f6457006c9ccbb2130916df6d34e4ef282c887843a1cc
Jul 21 00:20:55.174499 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.174413    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp"]
Jul 21 00:20:55.190166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.190112    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-f8n4k"]
Jul 21 00:20:55.192881 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:20:55.192854    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e48fe7f_30e5_402f_b4f5_ceaf629ca176.slice/crio-328c37cd15139d75137e55971d03288ec0f404809c941bb5f28bd4fd87981b96 WatchSource:0}: Error finding container 328c37cd15139d75137e55971d03288ec0f404809c941bb5f28bd4fd87981b96: Status 404 returned error can't find the container with id 328c37cd15139d75137e55971d03288ec0f404809c941bb5f28bd4fd87981b96
Jul 21 00:20:55.205859 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.205837    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-vqlfm"]
Jul 21 00:20:55.208055 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:20:55.208036    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c75a0d_5efe_450c_baff_342084431ad8.slice/crio-dc8cf26dee8bb869d66558a5cd95da074b78d655890731c2a62e0a268dca51b6 WatchSource:0}: Error finding container dc8cf26dee8bb869d66558a5cd95da074b78d655890731c2a62e0a268dca51b6: Status 404 returned error can't find the container with id dc8cf26dee8bb869d66558a5cd95da074b78d655890731c2a62e0a268dca51b6
Jul 21 00:20:55.252470 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.252423    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:55.252574 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.252534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:55.252574 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:55.252560    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:55.252669 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:55.252607    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:56.252592265 +0000 UTC m=+254.836455140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:55.252669 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:55.252659    2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 00:20:55.252738 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:55.252706    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls podName:a21bfe0c-4964-4d72-a133-0c4e8429aae7 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:56.252694334 +0000 UTC m=+254.836557209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zkxx6" (UID: "a21bfe0c-4964-4d72-a133-0c4e8429aae7") : secret "samples-operator-tls" not found
Jul 21 00:20:55.622716 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.622681    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7" event={"ID":"df455d46-57db-4f46-b9ab-75f218dd5aee","Type":"ContainerStarted","Data":"a351f7ea024a096021245ff40729980fc8c5c7b9e80db9846dbad07e0e889a0b"}
Jul 21 00:20:55.623737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.623710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-vqlfm" event={"ID":"18c75a0d-5efe-450c-baff-342084431ad8","Type":"ContainerStarted","Data":"dc8cf26dee8bb869d66558a5cd95da074b78d655890731c2a62e0a268dca51b6"}
Jul 21 00:20:55.624715 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.624694    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" event={"ID":"4e48fe7f-30e5-402f-b4f5-ceaf629ca176","Type":"ContainerStarted","Data":"328c37cd15139d75137e55971d03288ec0f404809c941bb5f28bd4fd87981b96"}
Jul 21 00:20:55.625621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.625606    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp" event={"ID":"0052f0d2-f154-44a7-aea7-40e1086be706","Type":"ContainerStarted","Data":"b6830d9f8f5410fa596f6457006c9ccbb2130916df6d34e4ef282c887843a1cc"}
Jul 21 00:20:55.626535 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:55.626518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck" event={"ID":"1b7c55fb-7095-405c-b7a9-bbbbef6d203c","Type":"ContainerStarted","Data":"798d5e7db0d398c0a93facacd9c3de3680144b9e937b76f57f4be9e652dcf90f"}
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:56.260574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:56.260678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:56.260831    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:56.260892    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:58.260874082 +0000 UTC m=+256.844736975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:56.261275    2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 00:20:56.261367 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:56.261327    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls podName:a21bfe0c-4964-4d72-a133-0c4e8429aae7 nodeName:}" failed. No retries permitted until 2026-07-21 00:20:58.261309915 +0000 UTC m=+256.845172795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zkxx6" (UID: "a21bfe0c-4964-4d72-a133-0c4e8429aae7") : secret "samples-operator-tls" not found
Jul 21 00:20:57.635230 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:57.635196    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck" event={"ID":"1b7c55fb-7095-405c-b7a9-bbbbef6d203c","Type":"ContainerStarted","Data":"82025f9fbfe77b01664b4e6ae76c8c3c9360f1b8bc6b866144a50e009448c09b"}
Jul 21 00:20:57.651844 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:57.651800    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-q7hck" podStartSLOduration=2.036630547 podStartE2EDuration="3.651784095s" totalImagesPullingTime="1.615153548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:20:57.650901535 +0000 UTC m=+256.234764424" watchObservedRunningTime="2026-07-21 00:20:57.651784095 +0000 UTC m=+256.235646993"
Jul 21 00:20:58.279112 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:58.279073    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:20:58.279281 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:58.279167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:20:58.279281 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:58.279258    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:58.279380 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:58.279342    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:02.279321181 +0000 UTC m=+260.863184059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:20:58.279380 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:58.279266    2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 00:20:58.279515 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:20:58.279407    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls podName:a21bfe0c-4964-4d72-a133-0c4e8429aae7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:02.279392917 +0000 UTC m=+260.863255792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zkxx6" (UID: "a21bfe0c-4964-4d72-a133-0c4e8429aae7") : secret "samples-operator-tls" not found
Jul 21 00:20:59.643331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.643285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7" event={"ID":"df455d46-57db-4f46-b9ab-75f218dd5aee","Type":"ContainerStarted","Data":"d10f7e955767713739cae9d07dc9c5faa88076183f05e581fa58edfc5db18c90"}
Jul 21 00:20:59.644921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.644890    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-vqlfm" event={"ID":"18c75a0d-5efe-450c-baff-342084431ad8","Type":"ContainerStarted","Data":"fabbfc1144ce6fe315fc0acf7bb23c5ff63b1a1fd91b38b01ad1a9276136be8f"}
Jul 21 00:20:59.646734 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.646710    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/0.log"
Jul 21 00:20:59.646872 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.646753    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e48fe7f-30e5-402f-b4f5-ceaf629ca176" containerID="5d923f1f838507a27787420e978d3c4a7f66b71105a6ba7c05e9a6a5ccf66398" exitCode=255
Jul 21 00:20:59.646872 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.646829    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" event={"ID":"4e48fe7f-30e5-402f-b4f5-ceaf629ca176","Type":"ContainerDied","Data":"5d923f1f838507a27787420e978d3c4a7f66b71105a6ba7c05e9a6a5ccf66398"}
Jul 21 00:20:59.647037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.647021    2561 scope.go:117] "RemoveContainer" containerID="5d923f1f838507a27787420e978d3c4a7f66b71105a6ba7c05e9a6a5ccf66398"
Jul 21 00:20:59.648400 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.648376    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp" event={"ID":"0052f0d2-f154-44a7-aea7-40e1086be706","Type":"ContainerStarted","Data":"79d85bb05bc90c7447fa5c032c4147c5a3580b105d6005634ccb90b5b00e5163"}
Jul 21 00:20:59.659187 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.659139    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7" podStartSLOduration=1.665231002 podStartE2EDuration="5.659122522s" totalImagesPullingTime="3.99389152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:20:59.658654858 +0000 UTC m=+258.242517759" watchObservedRunningTime="2026-07-21 00:20:59.659122522 +0000 UTC m=+258.242985423"
Jul 21 00:20:59.693401 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.691980    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-vqlfm" podStartSLOduration=1.809018994 podStartE2EDuration="5.691959782s" totalImagesPullingTime="3.882940788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:20:59.691708027 +0000 UTC m=+258.275570925" watchObservedRunningTime="2026-07-21 00:20:59.691959782 +0000 UTC m=+258.275822721"
Jul 21 00:20:59.693401 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:20:59.692531    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp" podStartSLOduration=1.7682667749999998 podStartE2EDuration="5.692518643s" totalImagesPullingTime="3.924251868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:20:59.674638547 +0000 UTC m=+258.258501446" watchObservedRunningTime="2026-07-21 00:20:59.692518643 +0000 UTC m=+258.276381541"
Jul 21 00:21:00.363630 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.363596    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"]
Jul 21 00:21:00.366951 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.366926    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"
Jul 21 00:21:00.369593 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.369569    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 21 00:21:00.370336 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.370311    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5t6k8\""
Jul 21 00:21:00.370471 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.370337    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 21 00:21:00.376492 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.376471    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"]
Jul 21 00:21:00.398728 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.398695    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqw98\" (UniqueName: \"kubernetes.io/projected/f1ea83d1-93fa-4f51-ac8a-e77d3767d64d-kube-api-access-xqw98\") pod \"migrator-7cc9c7ffd7-pm9gx\" (UID: \"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"
Jul 21 00:21:00.500163 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.500115    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqw98\" (UniqueName: \"kubernetes.io/projected/f1ea83d1-93fa-4f51-ac8a-e77d3767d64d-kube-api-access-xqw98\") pod \"migrator-7cc9c7ffd7-pm9gx\" (UID: \"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"
Jul 21 00:21:00.508342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.508315    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqw98\" (UniqueName: \"kubernetes.io/projected/f1ea83d1-93fa-4f51-ac8a-e77d3767d64d-kube-api-access-xqw98\") pod \"migrator-7cc9c7ffd7-pm9gx\" (UID: \"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"
Jul 21 00:21:00.609017 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.608979    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6sg77"]
Jul 21 00:21:00.614282 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.614208    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.616482 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.616461    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 00:21:00.616718 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.616700    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 00:21:00.616827 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.616748    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tx7qk\""
Jul 21 00:21:00.622851 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.622713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6sg77"]
Jul 21 00:21:00.653282 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653260    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:21:00.653657 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653643    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/0.log"
Jul 21 00:21:00.653696 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653677    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e48fe7f-30e5-402f-b4f5-ceaf629ca176" containerID="0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2" exitCode=255
Jul 21 00:21:00.653770 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653756    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" event={"ID":"4e48fe7f-30e5-402f-b4f5-ceaf629ca176","Type":"ContainerDied","Data":"0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2"}
Jul 21 00:21:00.653819 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653797    2561 scope.go:117] "RemoveContainer" containerID="5d923f1f838507a27787420e978d3c4a7f66b71105a6ba7c05e9a6a5ccf66398"
Jul 21 00:21:00.654007 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.653980    2561 scope.go:117] "RemoveContainer" containerID="0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2"
Jul 21 00:21:00.654236 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:00.654172    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-f8n4k_openshift-console-operator(4e48fe7f-30e5-402f-b4f5-ceaf629ca176)\"" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" podUID="4e48fe7f-30e5-402f-b4f5-ceaf629ca176"
Jul 21 00:21:00.676256 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.676229    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"
Jul 21 00:21:00.701542 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.701511    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.701755 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.701733    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scw5x\" (UniqueName: \"kubernetes.io/projected/440398bb-cc81-409c-92a2-127f84da31e7-kube-api-access-scw5x\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.701975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.701943    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/440398bb-cc81-409c-92a2-127f84da31e7-crio-socket\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.702250 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.702227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/440398bb-cc81-409c-92a2-127f84da31e7-data-volume\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.702323 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.702280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802402 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802362    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx"]
Jul 21 00:21:00.802666 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/440398bb-cc81-409c-92a2-127f84da31e7-crio-socket\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802718 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/440398bb-cc81-409c-92a2-127f84da31e7-data-volume\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802753 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802723    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802838 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802757    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/440398bb-cc81-409c-92a2-127f84da31e7-crio-socket\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802838 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802838 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:00.802816    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 00:21:00.802950 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.802844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scw5x\" (UniqueName: \"kubernetes.io/projected/440398bb-cc81-409c-92a2-127f84da31e7-kube-api-access-scw5x\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.802950 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:00.802881    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:01.30286275 +0000 UTC m=+259.886725629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : configmap "kube-rbac-proxy" not found
Jul 21 00:21:00.802950 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:00.802882    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:00.802950 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:00.802942    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:01.302929138 +0000 UTC m=+259.886792014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:00.803155 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.803070    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/440398bb-cc81-409c-92a2-127f84da31e7-data-volume\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:00.805085 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:00.805062    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ea83d1_93fa_4f51_ac8a_e77d3767d64d.slice/crio-242298e82c9f3a29e9c76170fba505c99b0a1cd0dabd81b24fba97cf2546f941 WatchSource:0}: Error finding container 242298e82c9f3a29e9c76170fba505c99b0a1cd0dabd81b24fba97cf2546f941: Status 404 returned error can't find the container with id 242298e82c9f3a29e9c76170fba505c99b0a1cd0dabd81b24fba97cf2546f941
Jul 21 00:21:00.810725 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:00.810704    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scw5x\" (UniqueName: \"kubernetes.io/projected/440398bb-cc81-409c-92a2-127f84da31e7-kube-api-access-scw5x\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:01.306388 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:01.306346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:01.306592 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:01.306406    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:01.306592 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:01.306534    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 00:21:01.306592 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:01.306537    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:01.306592 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:01.306588    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:02.306574366 +0000 UTC m=+260.890437241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : configmap "kube-rbac-proxy" not found
Jul 21 00:21:01.306732 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:01.306601    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:02.306595681 +0000 UTC m=+260.890458557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:01.658247 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:01.658167    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx" event={"ID":"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d","Type":"ContainerStarted","Data":"242298e82c9f3a29e9c76170fba505c99b0a1cd0dabd81b24fba97cf2546f941"}
Jul 21 00:21:01.659637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:01.659617    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:21:01.659968 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:01.659950    2561 scope.go:117] "RemoveContainer" containerID="0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2"
Jul 21 00:21:01.660115 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:01.660098    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-f8n4k_openshift-console-operator(4e48fe7f-30e5-402f-b4f5-ceaf629ca176)\"" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" podUID="4e48fe7f-30e5-402f-b4f5-ceaf629ca176"
Jul 21 00:21:02.317296 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.317254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:21:02.317501 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.317319    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:02.317501 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.317374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:02.317501 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.317415    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:21:02.317501 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317421    2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317508    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls podName:a21bfe0c-4964-4d72-a133-0c4e8429aae7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:10.317488509 +0000 UTC m=+268.901351408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zkxx6" (UID: "a21bfe0c-4964-4d72-a133-0c4e8429aae7") : secret "samples-operator-tls" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317512    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317530    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317559    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317575    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:04.317557333 +0000 UTC m=+262.901420242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317591    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:04.317583459 +0000 UTC m=+262.901446333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : configmap "kube-rbac-proxy" not found
Jul 21 00:21:02.317641 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:02.317601    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:10.317595305 +0000 UTC m=+268.901458179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:21:02.464404 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.464368    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v5nl8"]
Jul 21 00:21:02.466857 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.466840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.469751 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.469728    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 00:21:02.469844 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.469728    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 00:21:02.470584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.470561    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vknfq\""
Jul 21 00:21:02.470727 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.470602    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 00:21:02.470856 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.470825    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 00:21:02.476620 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.476600    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v5nl8"]
Jul 21 00:21:02.519123 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.519087    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/839962f0-c50d-4c84-8262-cc763b54800f-signing-key\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.519294 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.519128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/839962f0-c50d-4c84-8262-cc763b54800f-signing-cabundle\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.519294 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.519157    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftm79\" (UniqueName: \"kubernetes.io/projected/839962f0-c50d-4c84-8262-cc763b54800f-kube-api-access-ftm79\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.619866 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.619789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/839962f0-c50d-4c84-8262-cc763b54800f-signing-key\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.619866 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.619824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/839962f0-c50d-4c84-8262-cc763b54800f-signing-cabundle\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.619866 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.619845    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftm79\" (UniqueName: \"kubernetes.io/projected/839962f0-c50d-4c84-8262-cc763b54800f-kube-api-access-ftm79\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.620548 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.620527    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/839962f0-c50d-4c84-8262-cc763b54800f-signing-cabundle\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.622381 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.622361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/839962f0-c50d-4c84-8262-cc763b54800f-signing-key\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.627509 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.627484    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftm79\" (UniqueName: \"kubernetes.io/projected/839962f0-c50d-4c84-8262-cc763b54800f-kube-api-access-ftm79\") pod \"service-ca-58598695f7-v5nl8\" (UID: \"839962f0-c50d-4c84-8262-cc763b54800f\") " pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.664275 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.664245    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx" event={"ID":"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d","Type":"ContainerStarted","Data":"3e9df6ecda3703ac2f9be2eecf37be7fb9d51953d8d53d4acf18729199f356fb"}
Jul 21 00:21:02.664626 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.664300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx" event={"ID":"f1ea83d1-93fa-4f51-ac8a-e77d3767d64d","Type":"ContainerStarted","Data":"2556f6c5eec77eb1357cdc752b39917b4aff2043884aa7a5493813055a55cea5"}
Jul 21 00:21:02.680543 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.680496    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pm9gx" podStartSLOduration=1.504294716 podStartE2EDuration="2.680482954s" totalImagesPullingTime="1.176188238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:02.679677118 +0000 UTC m=+261.263540015" watchObservedRunningTime="2026-07-21 00:21:02.680482954 +0000 UTC m=+261.264345850"
Jul 21 00:21:02.775459 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.775409    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-v5nl8"
Jul 21 00:21:02.900062 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:02.900037    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v5nl8"]
Jul 21 00:21:02.902482 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:02.902430    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839962f0_c50d_4c84_8262_cc763b54800f.slice/crio-d65165071e20eefbb06e28f2bbb21dcd17d3878da446a0ca9ab7d4b3cc2fe214 WatchSource:0}: Error finding container d65165071e20eefbb06e28f2bbb21dcd17d3878da446a0ca9ab7d4b3cc2fe214: Status 404 returned error can't find the container with id d65165071e20eefbb06e28f2bbb21dcd17d3878da446a0ca9ab7d4b3cc2fe214
Jul 21 00:21:03.209730 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:03.209651    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m7f5t_ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665/dns-node-resolver/0.log"
Jul 21 00:21:03.669339 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:03.669299    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-v5nl8" event={"ID":"839962f0-c50d-4c84-8262-cc763b54800f","Type":"ContainerStarted","Data":"466064ddc8e2a8fd9db8304d512028ba1863cad423c5d8c2b2dade07e897293e"}
Jul 21 00:21:03.669837 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:03.669350    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-v5nl8" event={"ID":"839962f0-c50d-4c84-8262-cc763b54800f","Type":"ContainerStarted","Data":"d65165071e20eefbb06e28f2bbb21dcd17d3878da446a0ca9ab7d4b3cc2fe214"}
Jul 21 00:21:03.688129 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:03.688084    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-v5nl8" podStartSLOduration=1.6880649669999999 podStartE2EDuration="1.688064967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:03.687762345 +0000 UTC m=+262.271625238" watchObservedRunningTime="2026-07-21 00:21:03.688064967 +0000 UTC m=+262.271927866"
Jul 21 00:21:04.209262 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:04.209230    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kpx88_8a151b2d-b542-4a6b-99c7-5ce86b3be500/node-ca/0.log"
Jul 21 00:21:04.335411 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:04.335377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:04.335607 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:04.335450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:04.335607 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:04.335528    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 00:21:04.335607 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:04.335553    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:04.335607 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:04.335597    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:08.33557964 +0000 UTC m=+266.919442518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : configmap "kube-rbac-proxy" not found
Jul 21 00:21:04.335607 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:04.335611    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:08.335604937 +0000 UTC m=+266.919467812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:05.029108 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.029062    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:21:05.029706 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.029159    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:21:05.029706 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.029596    2561 scope.go:117] "RemoveContainer" containerID="0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2"
Jul 21 00:21:05.029834 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:05.029751    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-f8n4k_openshift-console-operator(4e48fe7f-30e5-402f-b4f5-ceaf629ca176)\"" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" podUID="4e48fe7f-30e5-402f-b4f5-ceaf629ca176"
Jul 21 00:21:05.408833 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.408747    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c778b_51ffe5ee-1c35-4eb8-a409-f3281f4b6797/serve-healthcheck-canary/0.log"
Jul 21 00:21:05.610396 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.610366    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pm9gx_f1ea83d1-93fa-4f51-ac8a-e77d3767d64d/migrator/0.log"
Jul 21 00:21:05.808867 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:05.808840    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pm9gx_f1ea83d1-93fa-4f51-ac8a-e77d3767d64d/graceful-termination/0.log"
Jul 21 00:21:06.011867 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:06.010721    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-sjjsp_0052f0d2-f154-44a7-aea7-40e1086be706/kube-storage-version-migrator-operator/0.log"
Jul 21 00:21:08.379976 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:08.379937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:08.380400 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:08.380021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:08.380400 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:08.380113    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:08.380400 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:08.380166    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 00:21:08.380400 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:08.380181    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:16.380165254 +0000 UTC m=+274.964028132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : secret "insights-runtime-extractor-tls" not found
Jul 21 00:21:08.380400 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:08.380227    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm podName:440398bb-cc81-409c-92a2-127f84da31e7 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:16.380210759 +0000 UTC m=+274.964073634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6sg77" (UID: "440398bb-cc81-409c-92a2-127f84da31e7") : configmap "kube-rbac-proxy" not found
Jul 21 00:21:10.400405 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:10.400368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:21:10.400848 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:10.400518    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:21:10.400848 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:10.400650    2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 00:21:10.400848 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:10.400729    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls podName:ebd52be5-0093-45e9-a9c5-4e89c42d1bd3 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:26.400707717 +0000 UTC m=+284.984570615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cbbjf" (UID: "ebd52be5-0093-45e9-a9c5-4e89c42d1bd3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 00:21:10.402980 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:10.402959    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a21bfe0c-4964-4d72-a133-0c4e8429aae7-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zkxx6\" (UID: \"a21bfe0c-4964-4d72-a133-0c4e8429aae7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:21:10.542034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:10.541978    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"
Jul 21 00:21:10.669110 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:10.669021    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6"]
Jul 21 00:21:11.699699 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:11.699659    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6" event={"ID":"a21bfe0c-4964-4d72-a133-0c4e8429aae7","Type":"ContainerStarted","Data":"2c0b5b2a840dc14058b374e5f6e8d2b5049988fee1b5c1001d00de15432b062d"}
Jul 21 00:21:13.706962 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:13.706929    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6" event={"ID":"a21bfe0c-4964-4d72-a133-0c4e8429aae7","Type":"ContainerStarted","Data":"d4762175ea775e3211deb5503f2798aabdd72118a40e2a83446a19c4f438a64d"}
Jul 21 00:21:13.707333 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:13.706985    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6" event={"ID":"a21bfe0c-4964-4d72-a133-0c4e8429aae7","Type":"ContainerStarted","Data":"1b85667b76ec893c3aec17debdfcf64933e2da04553086a11a287a38d258ac96"}
Jul 21 00:21:13.723322 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:13.723276    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zkxx6" podStartSLOduration=17.734164345 podStartE2EDuration="19.723260822s" totalImagesPullingTime="1.989096477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:13.723228878 +0000 UTC m=+272.307091775" watchObservedRunningTime="2026-07-21 00:21:13.723260822 +0000 UTC m=+272.307123709"
Jul 21 00:21:16.449381 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.449345    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:16.449871 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.449459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:16.449941 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.449920    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/440398bb-cc81-409c-92a2-127f84da31e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:16.451820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.451799    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/440398bb-cc81-409c-92a2-127f84da31e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6sg77\" (UID: \"440398bb-cc81-409c-92a2-127f84da31e7\") " pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:16.525063 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.525030    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6sg77"
Jul 21 00:21:16.667071 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.667047    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6sg77"]
Jul 21 00:21:16.669622 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:16.669593    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440398bb_cc81_409c_92a2_127f84da31e7.slice/crio-6ad27ad34379ea5495618e7ab089b11e7990e86e9b79fe9d0b1675e4c7e3f679 WatchSource:0}: Error finding container 6ad27ad34379ea5495618e7ab089b11e7990e86e9b79fe9d0b1675e4c7e3f679: Status 404 returned error can't find the container with id 6ad27ad34379ea5495618e7ab089b11e7990e86e9b79fe9d0b1675e4c7e3f679
Jul 21 00:21:16.716936 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:16.716857    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6sg77" event={"ID":"440398bb-cc81-409c-92a2-127f84da31e7","Type":"ContainerStarted","Data":"6ad27ad34379ea5495618e7ab089b11e7990e86e9b79fe9d0b1675e4c7e3f679"}
Jul 21 00:21:18.941653 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:18.941619    2561 scope.go:117] "RemoveContainer" containerID="0e7c24bd2447dd5f98cd1f37426ddf2f5f8dfdbbc5073eb907990938fe9c75e2"
Jul 21 00:21:21.738045 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:21.738016    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:21:21.738431 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:21.738087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" event={"ID":"4e48fe7f-30e5-402f-b4f5-ceaf629ca176","Type":"ContainerStarted","Data":"26e3a187e60c3dde74d92cad7679c8b1ed6089f0f243fd40e7470908a312b183"}
Jul 21 00:21:21.738431 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:21.738368    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:21:21.755411 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:21.755357    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k" podStartSLOduration=23.854707558 podStartE2EDuration="27.755341472s" totalImagesPullingTime="3.900633914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:21.754237384 +0000 UTC m=+280.338100283" watchObservedRunningTime="2026-07-21 00:21:21.755341472 +0000 UTC m=+280.339204368"
Jul 21 00:21:22.005570 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:22.005548    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-f8n4k"
Jul 21 00:21:22.747265 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:22.747221    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6sg77" event={"ID":"440398bb-cc81-409c-92a2-127f84da31e7","Type":"ContainerStarted","Data":"049329ce9fcb1751afa8b89388115d4b7343faa577c0e0f73d8f6625fd9c9185"}
Jul 21 00:21:23.395330 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:23.395248    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs" podUID="46bee106-bf85-4b5f-90a1-ef0cfafdcc86"
Jul 21 00:21:23.395330 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:23.395271    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d64586967-rvs5d" podUID="e987206e-2e14-4244-aee3-a4e8466d508b"
Jul 21 00:21:23.395330 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:23.395248    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9ctc" podUID="c3a03790-c202-4c90-9964-42b29b65a1d6"
Jul 21 00:21:23.751035 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:23.750955    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:23.751035 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:23.750969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:23.751035 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:23.750955    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:21:26.437041 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.436998    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:21:26.440003 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.439973    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebd52be5-0093-45e9-a9c5-4e89c42d1bd3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cbbjf\" (UID: \"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:21:26.449795 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.449770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"
Jul 21 00:21:26.593390 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.593366    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf"]
Jul 21 00:21:26.596041 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:26.596009    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd52be5_0093_45e9_a9c5_4e89c42d1bd3.slice/crio-46419bc9f812659c85d3b0db06594096bb4dd2cebfc3910f51e82355fe9de625 WatchSource:0}: Error finding container 46419bc9f812659c85d3b0db06594096bb4dd2cebfc3910f51e82355fe9de625: Status 404 returned error can't find the container with id 46419bc9f812659c85d3b0db06594096bb4dd2cebfc3910f51e82355fe9de625
Jul 21 00:21:26.764832 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.764781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6sg77" event={"ID":"440398bb-cc81-409c-92a2-127f84da31e7","Type":"ContainerStarted","Data":"f70b3bce6287b513a267b91b4a5ebf0ac64f14033e4e7570c263e786c10c0829"}
Jul 21 00:21:26.766121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:26.766087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf" event={"ID":"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3","Type":"ContainerStarted","Data":"46419bc9f812659c85d3b0db06594096bb4dd2cebfc3910f51e82355fe9de625"}
Jul 21 00:21:27.243911 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.243823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:21:27.243911 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.243882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:27.246832 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.246799    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46bee106-bf85-4b5f-90a1-ef0cfafdcc86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8ptrs\" (UID: \"46bee106-bf85-4b5f-90a1-ef0cfafdcc86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:21:27.246970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.246905    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"image-registry-d64586967-rvs5d\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") " pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:27.344980 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.344936    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:27.347804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.347780    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3a03790-c202-4c90-9964-42b29b65a1d6-metrics-tls\") pod \"dns-default-q9ctc\" (UID: \"c3a03790-c202-4c90-9964-42b29b65a1d6\") " pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:27.355422 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.355397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26lqt\""
Jul 21 00:21:27.355422 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.355415    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7r9b4\""
Jul 21 00:21:27.355598 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.355473    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hclsz\""
Jul 21 00:21:27.362718 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.362691    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"
Jul 21 00:21:27.362831 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.362738    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:27.362831 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.362789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:27.553606 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.553253    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9ctc"]
Jul 21 00:21:27.555405 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:27.555370    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a03790_c202_4c90_9964_42b29b65a1d6.slice/crio-7faa61f96187ac9e02648167325d71551a99d6a008270381ce8bde7115dc75f0 WatchSource:0}: Error finding container 7faa61f96187ac9e02648167325d71551a99d6a008270381ce8bde7115dc75f0: Status 404 returned error can't find the container with id 7faa61f96187ac9e02648167325d71551a99d6a008270381ce8bde7115dc75f0
Jul 21 00:21:27.772194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.772134    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9ctc" event={"ID":"c3a03790-c202-4c90-9964-42b29b65a1d6","Type":"ContainerStarted","Data":"7faa61f96187ac9e02648167325d71551a99d6a008270381ce8bde7115dc75f0"}
Jul 21 00:21:27.782239 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.782188    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:21:27.784130 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:27.784105    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs"]
Jul 21 00:21:27.784804 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:27.784776    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode987206e_2e14_4244_aee3_a4e8466d508b.slice/crio-06c1d4401614662c376c8be4040f6db7511d15877ea303e602f4965ddc70cb58 WatchSource:0}: Error finding container 06c1d4401614662c376c8be4040f6db7511d15877ea303e602f4965ddc70cb58: Status 404 returned error can't find the container with id 06c1d4401614662c376c8be4040f6db7511d15877ea303e602f4965ddc70cb58
Jul 21 00:21:27.787752 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:27.787727    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bee106_bf85_4b5f_90a1_ef0cfafdcc86.slice/crio-c4a2ace49c5b2eb80d5af36d3ecc1351c3a449b8235cc26329a03b7cc68e5ab8 WatchSource:0}: Error finding container c4a2ace49c5b2eb80d5af36d3ecc1351c3a449b8235cc26329a03b7cc68e5ab8: Status 404 returned error can't find the container with id c4a2ace49c5b2eb80d5af36d3ecc1351c3a449b8235cc26329a03b7cc68e5ab8
Jul 21 00:21:28.777692 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:28.777569    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d64586967-rvs5d" event={"ID":"e987206e-2e14-4244-aee3-a4e8466d508b","Type":"ContainerStarted","Data":"06c1d4401614662c376c8be4040f6db7511d15877ea303e602f4965ddc70cb58"}
Jul 21 00:21:28.778853 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:28.778826    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs" event={"ID":"46bee106-bf85-4b5f-90a1-ef0cfafdcc86","Type":"ContainerStarted","Data":"c4a2ace49c5b2eb80d5af36d3ecc1351c3a449b8235cc26329a03b7cc68e5ab8"}
Jul 21 00:21:30.788860 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.788819    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf" event={"ID":"ebd52be5-0093-45e9-a9c5-4e89c42d1bd3","Type":"ContainerStarted","Data":"77461e36ad5e165215f169d0a9abb8030ff47011a2986139c04697cf83c74198"}
Jul 21 00:21:30.791561 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.791210    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d64586967-rvs5d" event={"ID":"e987206e-2e14-4244-aee3-a4e8466d508b","Type":"ContainerStarted","Data":"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"}
Jul 21 00:21:30.791561 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.791279    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:30.793349 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.793319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs" event={"ID":"46bee106-bf85-4b5f-90a1-ef0cfafdcc86","Type":"ContainerStarted","Data":"bc30bf63e03f2175a10c97ec5f46b305fec3e12640658fec4c89b933db99f785"}
Jul 21 00:21:30.795802 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.795728    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6sg77" event={"ID":"440398bb-cc81-409c-92a2-127f84da31e7","Type":"ContainerStarted","Data":"39d125c9af9ad74e39697e980e4a15723be536aa0b93cf94224bcbf3c4f6b4fc"}
Jul 21 00:21:30.806317 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.805101    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cbbjf" podStartSLOduration=33.315919389 podStartE2EDuration="36.805082788s" totalImagesPullingTime="3.489163399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:30.804768498 +0000 UTC m=+289.388631433" watchObservedRunningTime="2026-07-21 00:21:30.805082788 +0000 UTC m=+289.388945687"
Jul 21 00:21:30.824119 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.823909    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6sg77" podStartSLOduration=17.564469457 podStartE2EDuration="30.823892948s" totalImagesPullingTime="13.259423491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 00:21:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:30.823342149 +0000 UTC m=+289.407205047" watchObservedRunningTime="2026-07-21 00:21:30.823892948 +0000 UTC m=+289.407755850"
Jul 21 00:21:30.840144 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.840050    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8ptrs" podStartSLOduration=280.039795164 podStartE2EDuration="4m42.84002955s" totalImagesPullingTime="2.800234386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:30.839194558 +0000 UTC m=+289.423057456" watchObservedRunningTime="2026-07-21 00:21:30.84002955 +0000 UTC m=+289.423892449"
Jul 21 00:21:30.860513 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:30.860460    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d64586967-rvs5d" podStartSLOduration=274.860422782 podStartE2EDuration="4m34.860422782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:16:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:30.858628963 +0000 UTC m=+289.442491894" watchObservedRunningTime="2026-07-21 00:21:30.860422782 +0000 UTC m=+289.444285679"
Jul 21 00:21:31.801746 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:31.801707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9ctc" event={"ID":"c3a03790-c202-4c90-9964-42b29b65a1d6","Type":"ContainerStarted","Data":"426150a8220c97664055fcbc722cc2719b09a3460d44f7156e7d3c947b72100b"}
Jul 21 00:21:31.802140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:31.801762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9ctc" event={"ID":"c3a03790-c202-4c90-9964-42b29b65a1d6","Type":"ContainerStarted","Data":"d9b31c74b4b260b3304b901925fece8d8c6ac2008792b4075c34f93fc9260c03"}
Jul 21 00:21:31.819530 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:31.819486    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q9ctc" podStartSLOduration=251.150914933 podStartE2EDuration="4m14.81947081s" totalImagesPullingTime="3.668555877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:31.818217103 +0000 UTC m=+290.402080002" watchObservedRunningTime="2026-07-21 00:21:31.81947081 +0000 UTC m=+290.403333737"
Jul 21 00:21:32.805178 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:32.805148    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:33.733293 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.733253    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"]
Jul 21 00:21:33.736505 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.736482    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:33.740921 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.740904    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 21 00:21:33.741030 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.740924    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 00:21:33.741030 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.740938    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-42qx7\""
Jul 21 00:21:33.741030 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.740973    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 21 00:21:33.748309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.748287    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"]
Jul 21 00:21:33.901483 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.901432    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:33.901894 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.901531    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:33.901894 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.901576    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/900771d1-6728-485d-b667-906657f967d8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:33.901894 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:33.901603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcfgf\" (UniqueName: \"kubernetes.io/projected/900771d1-6728-485d-b667-906657f967d8-kube-api-access-lcfgf\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.002329 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.002250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/900771d1-6728-485d-b667-906657f967d8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.002329 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.002281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcfgf\" (UniqueName: \"kubernetes.io/projected/900771d1-6728-485d-b667-906657f967d8-kube-api-access-lcfgf\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.002559 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.002332    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.002559 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.002363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.002936 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.002906    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/900771d1-6728-485d-b667-906657f967d8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.005064 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.005039    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.005160 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.005067    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/900771d1-6728-485d-b667-906657f967d8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.010071 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.010047    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcfgf\" (UniqueName: \"kubernetes.io/projected/900771d1-6728-485d-b667-906657f967d8-kube-api-access-lcfgf\") pod \"prometheus-operator-5945b5cfd-w2rfl\" (UID: \"900771d1-6728-485d-b667-906657f967d8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.045656 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.045632    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"
Jul 21 00:21:34.171942 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.171915    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl"]
Jul 21 00:21:34.174346 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:34.174318    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900771d1_6728_485d_b667_906657f967d8.slice/crio-cd1db15c08d380c40be2fad37b5b5395de2c5f0103ce18626275ddde59fe940a WatchSource:0}: Error finding container cd1db15c08d380c40be2fad37b5b5395de2c5f0103ce18626275ddde59fe940a: Status 404 returned error can't find the container with id cd1db15c08d380c40be2fad37b5b5395de2c5f0103ce18626275ddde59fe940a
Jul 21 00:21:34.814101 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:34.814061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl" event={"ID":"900771d1-6728-485d-b667-906657f967d8","Type":"ContainerStarted","Data":"cd1db15c08d380c40be2fad37b5b5395de2c5f0103ce18626275ddde59fe940a"}
Jul 21 00:21:35.818939 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:35.818915    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl" event={"ID":"900771d1-6728-485d-b667-906657f967d8","Type":"ContainerStarted","Data":"2475feec9b57bbcea2113a5f280450676f96e469e19eb6f6464861fb55dd8d72"}
Jul 21 00:21:36.823805 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:36.823763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl" event={"ID":"900771d1-6728-485d-b667-906657f967d8","Type":"ContainerStarted","Data":"329f7675ac61e7da10d7beac2d180e7b569c52e360a8d8e1f1d77e1f8af883b1"}
Jul 21 00:21:36.839948 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:36.839900    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w2rfl" podStartSLOduration=2.2621180069999998 podStartE2EDuration="3.839877293s" totalImagesPullingTime="1.577759286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:36.838621294 +0000 UTC m=+295.422484191" watchObservedRunningTime="2026-07-21 00:21:36.839877293 +0000 UTC m=+295.423740189"
Jul 21 00:21:37.942151 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:37.942112    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-987l2" podUID="d9d8c49a-18f0-46ae-af48-e8dcb11c3a55"
Jul 21 00:21:39.114193 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.114149    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-24nj8"]
Jul 21 00:21:39.117422 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.117404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.119779 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.119752    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zvvjz\""
Jul 21 00:21:39.119905 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.119829    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 00:21:39.120110 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.120090    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 00:21:39.120223 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.120106    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 00:21:39.149614 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149582    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-metrics-client-ca\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149623    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-accelerators-collector-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-sys\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149703    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-tls\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149735    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-wtmp\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149968 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149756    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149968 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149786    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-root\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149968 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149808    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-textfile\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.149968 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.149855    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qpw2\" (UniqueName: \"kubernetes.io/projected/456a8cb2-ef02-4ef9-879f-959435f4970c-kube-api-access-6qpw2\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250451 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-tls\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250451 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250453    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-wtmp\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-root\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250519    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-textfile\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qpw2\" (UniqueName: \"kubernetes.io/projected/456a8cb2-ef02-4ef9-879f-959435f4970c-kube-api-access-6qpw2\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250591    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-root\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-wtmp\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-metrics-client-ca\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.250681 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-accelerators-collector-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.251127 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250720    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-sys\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.251127 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/456a8cb2-ef02-4ef9-879f-959435f4970c-sys\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.251127 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.250916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-textfile\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.251276 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.251238    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-accelerators-collector-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.251417 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.251397    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456a8cb2-ef02-4ef9-879f-959435f4970c-metrics-client-ca\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.253019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.252990    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-tls\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.253019 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.253001    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456a8cb2-ef02-4ef9-879f-959435f4970c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.258427 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.258400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qpw2\" (UniqueName: \"kubernetes.io/projected/456a8cb2-ef02-4ef9-879f-959435f4970c-kube-api-access-6qpw2\") pod \"node-exporter-24nj8\" (UID: \"456a8cb2-ef02-4ef9-879f-959435f4970c\") " pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.428680 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.428588    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24nj8"
Jul 21 00:21:39.440120 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:39.440094    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456a8cb2_ef02_4ef9_879f_959435f4970c.slice/crio-a06bc04ef6560ddebfc4f3d28fd9d03d274e9ae65e2df97126675c86aaa0d54f WatchSource:0}: Error finding container a06bc04ef6560ddebfc4f3d28fd9d03d274e9ae65e2df97126675c86aaa0d54f: Status 404 returned error can't find the container with id a06bc04ef6560ddebfc4f3d28fd9d03d274e9ae65e2df97126675c86aaa0d54f
Jul 21 00:21:39.835667 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:39.835626    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24nj8" event={"ID":"456a8cb2-ef02-4ef9-879f-959435f4970c","Type":"ContainerStarted","Data":"a06bc04ef6560ddebfc4f3d28fd9d03d274e9ae65e2df97126675c86aaa0d54f"}
Jul 21 00:21:40.171039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.170340    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:21:40.175346 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.175323    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.177886 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.177840    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fmc2j\""
Jul 21 00:21:40.177886 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.177840    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 00:21:40.178056 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.177846    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 00:21:40.178146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178063    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 00:21:40.178146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178065    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 00:21:40.178272 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178258    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 00:21:40.178342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178274    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 00:21:40.178342 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178273    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 00:21:40.178573 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178558    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 00:21:40.178648 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.178623    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 00:21:40.192046 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.192025    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:21:40.260709 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260679    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.260893 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260746    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.260893 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260774    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.260893 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.260893 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260858    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.260893 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260880    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260904    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hhv5\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.260949    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.261042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.261083    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261258 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.261125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261258 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.261152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.261258 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.261182    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362336 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362301    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362347    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362407    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hhv5\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362536    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362570    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362592    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362642    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362669    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.362861 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.362700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.363155 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:21:40.362930    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle podName:4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5 nodeName:}" failed. No retries permitted until 2026-07-21 00:21:40.862905653 +0000 UTC m=+299.446768528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5") : configmap references non-existent config key: ca-bundle.crt
Jul 21 00:21:40.363929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.363867    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.364166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.364128    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.365747 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.365719    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.366930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.366894    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.366972    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.367004    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367155 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.367085    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367391 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.367360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367661 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.367644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.367807 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.367787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.368121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.368098    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.369937 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.369910    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hhv5\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.868131 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.868104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:40.870491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:40.869846    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:41.085920 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.085839    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:21:41.223798 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.223770    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:21:41.224933 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:41.224904    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1b1e9e_d0d9_48d6_9ad7_f47a9e2d95b5.slice/crio-fcb4b3af0b569a28b88922db643bf54e09518aaa062bb708d19fcc251be1f5cf WatchSource:0}: Error finding container fcb4b3af0b569a28b88922db643bf54e09518aaa062bb708d19fcc251be1f5cf: Status 404 returned error can't find the container with id fcb4b3af0b569a28b88922db643bf54e09518aaa062bb708d19fcc251be1f5cf
Jul 21 00:21:41.846475 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.846292    2561 generic.go:356] "Generic (PLEG): container finished" podID="456a8cb2-ef02-4ef9-879f-959435f4970c" containerID="57bc6938ed6075159b148d3846e4ea8e9468acb824680b2c869c373a33b4b5d2" exitCode=0
Jul 21 00:21:41.846475 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.846406    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24nj8" event={"ID":"456a8cb2-ef02-4ef9-879f-959435f4970c","Type":"ContainerDied","Data":"57bc6938ed6075159b148d3846e4ea8e9468acb824680b2c869c373a33b4b5d2"}
Jul 21 00:21:41.849656 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.849603    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"fcb4b3af0b569a28b88922db643bf54e09518aaa062bb708d19fcc251be1f5cf"}
Jul 21 00:21:41.852905 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.852881    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:21:41.853388 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.853366    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:21:41.877026 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:41.876994    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 00:21:42.810746 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.810715    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q9ctc"
Jul 21 00:21:42.856113 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.856065    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24nj8" event={"ID":"456a8cb2-ef02-4ef9-879f-959435f4970c","Type":"ContainerStarted","Data":"1e0a91ccb5f5da7df9e5bfa724aed77be762513949095b54bcff6da77ac9852c"}
Jul 21 00:21:42.856280 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.856147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24nj8" event={"ID":"456a8cb2-ef02-4ef9-879f-959435f4970c","Type":"ContainerStarted","Data":"65cdac6936e261a6ae4d82fcbe501cedd78d5f97ed5f7420be2dcaaa96993938"}
Jul 21 00:21:42.857659 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.857627    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d" exitCode=0
Jul 21 00:21:42.857796 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.857687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"}
Jul 21 00:21:42.876722 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:42.876681    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-24nj8" podStartSLOduration=2.443870957 podStartE2EDuration="3.876661925s" totalImagesPullingTime="1.350932976s" totalInitContainerRuntime="81.857992ms" isStatefulPod=true podCreationTimestamp="2026-07-21 00:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:42.876014266 +0000 UTC m=+301.459877185" watchObservedRunningTime="2026-07-21 00:21:42.876661925 +0000 UTC m=+301.460524825"
Jul 21 00:21:43.087474 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.087376    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:21:43.090632 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.090611    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.093364 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093343    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 00:21:43.093364 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093356    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 00:21:43.093601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093580    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ks6p9\""
Jul 21 00:21:43.093691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093602    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 00:21:43.093691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093603    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 00:21:43.093691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093676    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 00:21:43.093691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093677    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 00:21:43.093957 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.093915    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 00:21:43.099006 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.098984    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 00:21:43.100034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.100013    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:21:43.190140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190236    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190267    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.190481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.190330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fmlh\" (UniqueName: \"kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291197    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fmlh\" (UniqueName: \"kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291271    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291296    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.291488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.291381    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.292086 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.292056    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.292314 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.292145    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.292314 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.292248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.292314 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.292248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.294040 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.294019    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.294121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.294019    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.299768 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.299745    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fmlh\" (UniqueName: \"kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh\") pod \"console-56454dffc-gjwbk\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") " pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.400912 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.400835    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:43.551706 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.551684    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:21:43.554139 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:43.554112    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3a5b22_bbac_4d45_b63d_98df62afb07a.slice/crio-87dedabbbb6088d415ec757282ad8407c796371121a4670479c1ced3426bb118 WatchSource:0}: Error finding container 87dedabbbb6088d415ec757282ad8407c796371121a4670479c1ced3426bb118: Status 404 returned error can't find the container with id 87dedabbbb6088d415ec757282ad8407c796371121a4670479c1ced3426bb118
Jul 21 00:21:43.556220 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.556204    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 00:21:43.863682 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:43.863648    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56454dffc-gjwbk" event={"ID":"4f3a5b22-bbac-4d45-b63d-98df62afb07a","Type":"ContainerStarted","Data":"87dedabbbb6088d415ec757282ad8407c796371121a4670479c1ced3426bb118"}
Jul 21 00:21:44.376665 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.376624    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c56898f4f-rw6r7"]
Jul 21 00:21:44.380793 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.380767    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.383636 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383607    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 21 00:21:44.383770 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383676    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 21 00:21:44.383770 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383720    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 21 00:21:44.383882 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383859    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 21 00:21:44.383971 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383955    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 21 00:21:44.384024 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.383955    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2fgqj\""
Jul 21 00:21:44.391033 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.390873    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 21 00:21:44.395778 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.395757    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c56898f4f-rw6r7"]
Jul 21 00:21:44.401990 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.401968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-serving-certs-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-federate-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402084    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402281 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402146    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402281 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-metrics-client-ca\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402281 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.402281 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.402271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdlk\" (UniqueName: \"kubernetes.io/projected/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-kube-api-access-jfdlk\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503042 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-metrics-client-ca\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503079    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdlk\" (UniqueName: \"kubernetes.io/projected/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-kube-api-access-jfdlk\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503121    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-serving-certs-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503148    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-federate-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503171    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503229 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.503949 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.503917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-metrics-client-ca\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.504396 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.504344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-serving-certs-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.504647 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.504620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.506580 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.506555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-telemeter-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.506908 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.506883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-federate-client-tls\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.507050 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.507008    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.507711 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.507691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.511195 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.511175    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdlk\" (UniqueName: \"kubernetes.io/projected/fbd21c1e-c5ba-4033-aaeb-a6f9410106c8-kube-api-access-jfdlk\") pod \"telemeter-client-c56898f4f-rw6r7\" (UID: \"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8\") " pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.693690 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.693607    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7"
Jul 21 00:21:44.898979 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:44.898953    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c56898f4f-rw6r7"]
Jul 21 00:21:44.901859 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:44.901824    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd21c1e_c5ba_4033_aaeb_a6f9410106c8.slice/crio-5b3d0d5828d01c9a1ec83a442b5fca2c01146770faff5acaba32597ac28e3536 WatchSource:0}: Error finding container 5b3d0d5828d01c9a1ec83a442b5fca2c01146770faff5acaba32597ac28e3536: Status 404 returned error can't find the container with id 5b3d0d5828d01c9a1ec83a442b5fca2c01146770faff5acaba32597ac28e3536
Jul 21 00:21:45.876652 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.876612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"}
Jul 21 00:21:45.876833 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.876729    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"}
Jul 21 00:21:45.876833 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.876773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"}
Jul 21 00:21:45.876833 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.876809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"}
Jul 21 00:21:45.876992 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.876844    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"}
Jul 21 00:21:45.878003 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:45.877973    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7" event={"ID":"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8","Type":"ContainerStarted","Data":"5b3d0d5828d01c9a1ec83a442b5fca2c01146770faff5acaba32597ac28e3536"}
Jul 21 00:21:46.884851 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:46.884662    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56454dffc-gjwbk" event={"ID":"4f3a5b22-bbac-4d45-b63d-98df62afb07a","Type":"ContainerStarted","Data":"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"}
Jul 21 00:21:46.906218 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:46.906111    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56454dffc-gjwbk" podStartSLOduration=0.813867934 podStartE2EDuration="3.906094673s" totalImagesPullingTime="3.092226739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:46.903604729 +0000 UTC m=+305.487467649" watchObservedRunningTime="2026-07-21 00:21:46.906094673 +0000 UTC m=+305.489957569"
Jul 21 00:21:47.367800 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.367762    2561 patch_prober.go:28] interesting pod/image-registry-d64586967-rvs5d container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 21 00:21:47.367976 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.367822    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d64586967-rvs5d" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 00:21:47.495393 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.495361    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:21:47.890579 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.890483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7" event={"ID":"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8","Type":"ContainerStarted","Data":"744a52c7bacbba31a2656b6c2be5491213919e8de06b33d18d9404ec52b5b1a5"}
Jul 21 00:21:47.890579 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.890551    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7" event={"ID":"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8","Type":"ContainerStarted","Data":"1df8071d17344738baa72f7a66098636b602840fbf8aa2ffd0cbb75b6b0aaa89"}
Jul 21 00:21:47.891079 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.890583    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7" event={"ID":"fbd21c1e-c5ba-4033-aaeb-a6f9410106c8","Type":"ContainerStarted","Data":"b8161d0f5cfc2c3683eff40101b02b6e70e9e148498a11d67dd18f55816fe095"}
Jul 21 00:21:47.893847 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.893808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerStarted","Data":"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"}
Jul 21 00:21:47.911407 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.911369    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c56898f4f-rw6r7" podStartSLOduration=1.271745461 podStartE2EDuration="3.911354859s" totalImagesPullingTime="2.639609398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:47.909784788 +0000 UTC m=+306.493647685" watchObservedRunningTime="2026-07-21 00:21:47.911354859 +0000 UTC m=+306.495217755"
Jul 21 00:21:47.937075 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:47.937030    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.057762078 podStartE2EDuration="7.937014103s" totalImagesPullingTime="4.860633629s" totalInitContainerRuntime="18.618396ms" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:47.935665435 +0000 UTC m=+306.519528357" watchObservedRunningTime="2026-07-21 00:21:47.937014103 +0000 UTC m=+306.520876999"
Jul 21 00:21:48.731823 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.731780    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:21:48.735632 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.735605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.744645 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.744618    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:21:48.848298 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848266    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848315    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848332    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848481 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848384    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjbg\" (UniqueName: \"kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848655 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.848655 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.848567    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.949300 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.949266    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.949700 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.949622    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.949755 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.949721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.949755 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.949749    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.950008 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.949988    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.950120 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.950022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjbg\" (UniqueName: \"kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.950184 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.950144    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.950822 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.950385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.950944 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.950917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.951043 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.951026    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.951288 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.951230    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.952269 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.952243    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.952993 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.952968    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:48.958235 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:48.958210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjbg\" (UniqueName: \"kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg\") pod \"console-68bbddf9cc-5jhgw\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") " pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:49.047801 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:49.047768    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:49.177478 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:49.177428    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:21:49.180172 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:21:49.180147    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1628e45b_f93f_470d_bd15_12ee8bf8199f.slice/crio-9fdb031a56b07d7dfd6c8d8dde6b337194ab7998796ef5dccd2efa21f5af0329 WatchSource:0}: Error finding container 9fdb031a56b07d7dfd6c8d8dde6b337194ab7998796ef5dccd2efa21f5af0329: Status 404 returned error can't find the container with id 9fdb031a56b07d7dfd6c8d8dde6b337194ab7998796ef5dccd2efa21f5af0329
Jul 21 00:21:49.905352 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:49.905318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bbddf9cc-5jhgw" event={"ID":"1628e45b-f93f-470d-bd15-12ee8bf8199f","Type":"ContainerStarted","Data":"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"}
Jul 21 00:21:49.905528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:49.905374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bbddf9cc-5jhgw" event={"ID":"1628e45b-f93f-470d-bd15-12ee8bf8199f","Type":"ContainerStarted","Data":"9fdb031a56b07d7dfd6c8d8dde6b337194ab7998796ef5dccd2efa21f5af0329"}
Jul 21 00:21:49.922616 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:49.922569    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68bbddf9cc-5jhgw" podStartSLOduration=1.922552888 podStartE2EDuration="1.922552888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:21:49.920527496 +0000 UTC m=+308.504390405" watchObservedRunningTime="2026-07-21 00:21:49.922552888 +0000 UTC m=+308.506415784"
Jul 21 00:21:50.941488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:50.941461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:21:51.806529 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:51.806500    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:21:53.023003 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:53.022968    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:21:53.401608 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:53.401495    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:21:59.048831 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:59.048795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:59.049362 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:59.048845    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:59.053643 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:59.053623    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:21:59.948533 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:21:59.948507    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:22:13.918951 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:13.918881    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56454dffc-gjwbk" podUID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" containerName="console" containerID="cri-o://e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9" gracePeriod=15
Jul 21 00:22:14.159375 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.159353    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:22:14.276114 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276083    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276124    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276143    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276241    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fmlh\" (UniqueName: \"kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276364    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca\") pod \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\" (UID: \"4f3a5b22-bbac-4d45-b63d-98df62afb07a\") "
Jul 21 00:22:14.276695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276635    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config" (OuterVolumeSpecName: "console-config") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:14.276695 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276656    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:14.276797 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276701    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:14.276902 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.276876    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:14.277044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.277020    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-trusted-ca-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.277044 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.277045    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-oauth-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.277593 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.277061    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.278612 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.278588    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:22:14.278708 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.278650    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh" (OuterVolumeSpecName: "kube-api-access-8fmlh") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "kube-api-access-8fmlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:22:14.278785 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.278758    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f3a5b22-bbac-4d45-b63d-98df62afb07a" (UID: "4f3a5b22-bbac-4d45-b63d-98df62afb07a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:22:14.378194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.378159    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fmlh\" (UniqueName: \"kubernetes.io/projected/4f3a5b22-bbac-4d45-b63d-98df62afb07a-kube-api-access-8fmlh\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.378194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.378188    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3a5b22-bbac-4d45-b63d-98df62afb07a-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.378194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.378198    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-oauth-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:14.378410 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:14.378206    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3a5b22-bbac-4d45-b63d-98df62afb07a-console-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:15.001091 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.001058    2561 generic.go:356] "Generic (PLEG): container finished" podID="0052f0d2-f154-44a7-aea7-40e1086be706" containerID="79d85bb05bc90c7447fa5c032c4147c5a3580b105d6005634ccb90b5b00e5163" exitCode=0
Jul 21 00:22:15.001482 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.001132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp" event={"ID":"0052f0d2-f154-44a7-aea7-40e1086be706","Type":"ContainerDied","Data":"79d85bb05bc90c7447fa5c032c4147c5a3580b105d6005634ccb90b5b00e5163"}
Jul 21 00:22:15.001540 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.001528    2561 scope.go:117] "RemoveContainer" containerID="79d85bb05bc90c7447fa5c032c4147c5a3580b105d6005634ccb90b5b00e5163"
Jul 21 00:22:15.002515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.002429    2561 generic.go:356] "Generic (PLEG): container finished" podID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" containerID="e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9" exitCode=0
Jul 21 00:22:15.002515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.002488    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56454dffc-gjwbk" event={"ID":"4f3a5b22-bbac-4d45-b63d-98df62afb07a","Type":"ContainerDied","Data":"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"}
Jul 21 00:22:15.002515 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.002497    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56454dffc-gjwbk"
Jul 21 00:22:15.002671 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.002529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56454dffc-gjwbk" event={"ID":"4f3a5b22-bbac-4d45-b63d-98df62afb07a","Type":"ContainerDied","Data":"87dedabbbb6088d415ec757282ad8407c796371121a4670479c1ced3426bb118"}
Jul 21 00:22:15.002671 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.002554    2561 scope.go:117] "RemoveContainer" containerID="e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"
Jul 21 00:22:15.012204 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.012188    2561 scope.go:117] "RemoveContainer" containerID="e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"
Jul 21 00:22:15.012514 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:22:15.012489    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9\": container with ID starting with e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9 not found: ID does not exist" containerID="e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"
Jul 21 00:22:15.012601 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.012525    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9"} err="failed to get container status \"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9\": rpc error: code = NotFound desc = could not find container \"e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9\": container with ID starting with e58e7cbcd8a48fc825eac1b0b17ad2d4716d73b64355f57f7ac4d5b029de8df9 not found: ID does not exist"
Jul 21 00:22:15.031514 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.031488    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:22:15.037421 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.037398    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56454dffc-gjwbk"]
Jul 21 00:22:15.946039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:15.945996    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" path="/var/lib/kubelet/pods/4f3a5b22-bbac-4d45-b63d-98df62afb07a/volumes"
Jul 21 00:22:16.008586 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:16.008554    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-sjjsp" event={"ID":"0052f0d2-f154-44a7-aea7-40e1086be706","Type":"ContainerStarted","Data":"ad42ed0cc1dd82b7143fd0bf3dd673f1b79b3ae5220e6be65c1cbab8d9d26aac"}
Jul 21 00:22:18.042823 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.042776    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d64586967-rvs5d" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" containerName="registry" containerID="cri-o://a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c" gracePeriod=30
Jul 21 00:22:18.307089 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.307027    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:22:18.416352 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416321    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416379    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416416    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416481    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416508    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416552 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416529    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k57fh\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416791 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416558    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.416791 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.416585    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted\") pod \"e987206e-2e14-4244-aee3-a4e8466d508b\" (UID: \"e987206e-2e14-4244-aee3-a4e8466d508b\") "
Jul 21 00:22:18.417078 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.417026    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:18.417183 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.417139    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:22:18.419395 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.419356    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh" (OuterVolumeSpecName: "kube-api-access-k57fh") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "kube-api-access-k57fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:22:18.419678 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.419652    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:22:18.419871 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.419840    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:22:18.419871 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.419853    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:22:18.420092 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.420074    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:22:18.425291 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.425269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e987206e-2e14-4244-aee3-a4e8466d508b" (UID: "e987206e-2e14-4244-aee3-a4e8466d508b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:22:18.517384 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517353    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-registry-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517384 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517378    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k57fh\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-kube-api-access-k57fh\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517384 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517389    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e987206e-2e14-4244-aee3-a4e8466d508b-bound-sa-token\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517400    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e987206e-2e14-4244-aee3-a4e8466d508b-ca-trust-extracted\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517409    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-installation-pull-secrets\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517419    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e987206e-2e14-4244-aee3-a4e8466d508b-image-registry-private-configuration\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517428    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-trusted-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:18.517633 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:18.517452    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e987206e-2e14-4244-aee3-a4e8466d508b-registry-certificates\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:22:19.021069 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.021038    2561 generic.go:356] "Generic (PLEG): container finished" podID="e987206e-2e14-4244-aee3-a4e8466d508b" containerID="a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c" exitCode=0
Jul 21 00:22:19.021245 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.021086    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d64586967-rvs5d"
Jul 21 00:22:19.021245 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.021102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d64586967-rvs5d" event={"ID":"e987206e-2e14-4244-aee3-a4e8466d508b","Type":"ContainerDied","Data":"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"}
Jul 21 00:22:19.021245 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.021143    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d64586967-rvs5d" event={"ID":"e987206e-2e14-4244-aee3-a4e8466d508b","Type":"ContainerDied","Data":"06c1d4401614662c376c8be4040f6db7511d15877ea303e602f4965ddc70cb58"}
Jul 21 00:22:19.021245 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.021163    2561 scope.go:117] "RemoveContainer" containerID="a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"
Jul 21 00:22:19.034038 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.034018    2561 scope.go:117] "RemoveContainer" containerID="a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"
Jul 21 00:22:19.034329 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:22:19.034308    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c\": container with ID starting with a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c not found: ID does not exist" containerID="a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"
Jul 21 00:22:19.034409 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.034337    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c"} err="failed to get container status \"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c\": rpc error: code = NotFound desc = could not find container \"a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c\": container with ID starting with a9a406aa32d15616638e121c765c061d345febd9d1ee999383174fdb223c7f2c not found: ID does not exist"
Jul 21 00:22:19.041211 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.041185    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:22:19.044717 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.044699    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d64586967-rvs5d"]
Jul 21 00:22:19.946673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:19.946631    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" path="/var/lib/kubelet/pods/e987206e-2e14-4244-aee3-a4e8466d508b/volumes"
Jul 21 00:22:20.026697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:20.026665    2561 generic.go:356] "Generic (PLEG): container finished" podID="df455d46-57db-4f46-b9ab-75f218dd5aee" containerID="d10f7e955767713739cae9d07dc9c5faa88076183f05e581fa58edfc5db18c90" exitCode=0
Jul 21 00:22:20.026854 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:20.026727    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7" event={"ID":"df455d46-57db-4f46-b9ab-75f218dd5aee","Type":"ContainerDied","Data":"d10f7e955767713739cae9d07dc9c5faa88076183f05e581fa58edfc5db18c90"}
Jul 21 00:22:20.027045 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:20.027032    2561 scope.go:117] "RemoveContainer" containerID="d10f7e955767713739cae9d07dc9c5faa88076183f05e581fa58edfc5db18c90"
Jul 21 00:22:21.035274 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:21.035219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-f4gc7" event={"ID":"df455d46-57db-4f46-b9ab-75f218dd5aee","Type":"ContainerStarted","Data":"8ff6a7c926b04f3bed9b581e41497b9afed22a2a60791f070f843741cbf84dd1"}
Jul 21 00:22:30.073453 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:30.073404    2561 generic.go:356] "Generic (PLEG): container finished" podID="18c75a0d-5efe-450c-baff-342084431ad8" containerID="fabbfc1144ce6fe315fc0acf7bb23c5ff63b1a1fd91b38b01ad1a9276136be8f" exitCode=0
Jul 21 00:22:30.073884 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:30.073473    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-vqlfm" event={"ID":"18c75a0d-5efe-450c-baff-342084431ad8","Type":"ContainerDied","Data":"fabbfc1144ce6fe315fc0acf7bb23c5ff63b1a1fd91b38b01ad1a9276136be8f"}
Jul 21 00:22:30.073884 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:30.073798    2561 scope.go:117] "RemoveContainer" containerID="fabbfc1144ce6fe315fc0acf7bb23c5ff63b1a1fd91b38b01ad1a9276136be8f"
Jul 21 00:22:30.964666 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:30.964637    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c778b_51ffe5ee-1c35-4eb8-a409-f3281f4b6797/serve-healthcheck-canary/0.log"
Jul 21 00:22:31.079274 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:31.079244    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-vqlfm" event={"ID":"18c75a0d-5efe-450c-baff-342084431ad8","Type":"ContainerStarted","Data":"87687f67176e8b298c1d2d35e42cf82f6206bd6ea4c501d7436116045ca4f5b0"}
Jul 21 00:22:55.848173 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:55.848124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:22:55.850598 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:55.850578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d8c49a-18f0-46ae-af48-e8dcb11c3a55-metrics-certs\") pod \"network-metrics-daemon-987l2\" (UID: \"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55\") " pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:22:56.045145 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:56.045111    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztz7f\""
Jul 21 00:22:56.052604 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:56.052586    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-987l2"
Jul 21 00:22:56.183313 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:56.183283    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-987l2"]
Jul 21 00:22:56.189627 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:22:56.189596    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d8c49a_18f0_46ae_af48_e8dcb11c3a55.slice/crio-140abdefd48643268d15677bd51a0a1e4b2d14a41984642bbbaa97051de9e28b WatchSource:0}: Error finding container 140abdefd48643268d15677bd51a0a1e4b2d14a41984642bbbaa97051de9e28b: Status 404 returned error can't find the container with id 140abdefd48643268d15677bd51a0a1e4b2d14a41984642bbbaa97051de9e28b
Jul 21 00:22:57.192740 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:57.192708    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-987l2" event={"ID":"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55","Type":"ContainerStarted","Data":"140abdefd48643268d15677bd51a0a1e4b2d14a41984642bbbaa97051de9e28b"}
Jul 21 00:22:58.198292 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:58.198256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-987l2" event={"ID":"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55","Type":"ContainerStarted","Data":"21c2ea5e63bcd4d06b4be7dde215ad50dc7a22949813b872351313b711f0e92d"}
Jul 21 00:22:58.198804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:58.198318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-987l2" event={"ID":"d9d8c49a-18f0-46ae-af48-e8dcb11c3a55","Type":"ContainerStarted","Data":"502479dbdb91b7a7ff7108048ac1bfae7abcd5cd3b5b7b4030aa5ea42a272deb"}
Jul 21 00:22:58.216503 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:58.216431    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-987l2" podStartSLOduration=375.082849354 podStartE2EDuration="6m16.216415493s" totalImagesPullingTime="1.133566139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:22:58.215425925 +0000 UTC m=+376.799288823" watchObservedRunningTime="2026-07-21 00:22:58.216415493 +0000 UTC m=+376.800278389"
Jul 21 00:22:59.445164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445106    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445376    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="alertmanager" containerID="cri-o://544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526" gracePeriod=120
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445467    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-metric" containerID="cri-o://ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe" gracePeriod=120
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445486    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-web" containerID="cri-o://1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c" gracePeriod=120
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445506    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="config-reloader" containerID="cri-o://8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305" gracePeriod=120
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445538    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy" containerID="cri-o://50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4" gracePeriod=120
Jul 21 00:22:59.445621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:22:59.445604    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="prom-label-proxy" containerID="cri-o://af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422" gracePeriod=120
Jul 21 00:23:00.209528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209490    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422" exitCode=0
Jul 21 00:23:00.209528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209515    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4" exitCode=0
Jul 21 00:23:00.209528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209521    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305" exitCode=0
Jul 21 00:23:00.209528 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209526    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526" exitCode=0
Jul 21 00:23:00.209852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209566    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"}
Jul 21 00:23:00.209852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209649    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"}
Jul 21 00:23:00.209852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"}
Jul 21 00:23:00.209852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.209723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"}
Jul 21 00:23:00.691625 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.691606    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:00.784986 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.784951    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.784991    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hhv5\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785019    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785048    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785071    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785090    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785148    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785172    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785213    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785252 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785242    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785273    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785304    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785355    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out\") pod \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\" (UID: \"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5\") "
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785515    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785515    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785675    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-main-db\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.785918 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.785694    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-metrics-client-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.786293 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.786267    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:00.788801 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.788762    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.788925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.788841    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.788925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.788862    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.789062 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.788980    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5" (OuterVolumeSpecName: "kube-api-access-4hhv5") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "kube-api-access-4hhv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:23:00.789312 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.789290    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out" (OuterVolumeSpecName: "config-out") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:23:00.789729 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.789707    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.789876 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.789858    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:23:00.790587 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.790564    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume" (OuterVolumeSpecName: "config-volume") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.793739 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.793613    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.800646 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.800623    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config" (OuterVolumeSpecName: "web-config") pod "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" (UID: "4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:00.886958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886921    2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-cluster-tls-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.886958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886951    2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-volume\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.886958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886960    2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-tls-assets\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.886958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886970    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-main-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886979    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886990    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.886999    2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-config-out\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.887007    2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-web-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.887014    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hhv5\" (UniqueName: \"kubernetes.io/projected/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-kube-api-access-4hhv5\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.887023    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:00.887209 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:00.887033    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:01.216491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216392    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe" exitCode=0
Jul 21 00:23:01.216491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216418    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerID="1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c" exitCode=0
Jul 21 00:23:01.216491 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216477    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"}
Jul 21 00:23:01.216689 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216492    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.216689 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216551    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"}
Jul 21 00:23:01.216689 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216579    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5","Type":"ContainerDied","Data":"fcb4b3af0b569a28b88922db643bf54e09518aaa062bb708d19fcc251be1f5cf"}
Jul 21 00:23:01.216689 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.216619    2561 scope.go:117] "RemoveContainer" containerID="af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"
Jul 21 00:23:01.224879 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.224858    2561 scope.go:117] "RemoveContainer" containerID="ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"
Jul 21 00:23:01.232030 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.232005    2561 scope.go:117] "RemoveContainer" containerID="50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"
Jul 21 00:23:01.238896 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.238870    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:23:01.239503 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.239483    2561 scope.go:117] "RemoveContainer" containerID="1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"
Jul 21 00:23:01.243151 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.243132    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:23:01.248357 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.248335    2561 scope.go:117] "RemoveContainer" containerID="8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"
Jul 21 00:23:01.255418 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.255391    2561 scope.go:117] "RemoveContainer" containerID="544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"
Jul 21 00:23:01.262686 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.262670    2561 scope.go:117] "RemoveContainer" containerID="603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"
Jul 21 00:23:01.267962 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.267942    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:23:01.268230 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268219    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="prom-label-proxy"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268232    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="prom-label-proxy"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268244    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="init-config-reloader"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268250    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="init-config-reloader"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268259    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="alertmanager"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268264    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="alertmanager"
Jul 21 00:23:01.268270 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268270    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-web"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268275    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-web"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268281    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-metric"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268287    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-metric"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268299    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="config-reloader"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268304    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="config-reloader"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268310    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268316    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268325    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" containerName="console"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268329    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" containerName="console"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268338    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" containerName="registry"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268343    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" containerName="registry"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268394    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-web"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268403    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="config-reloader"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268410    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e987206e-2e14-4244-aee3-a4e8466d508b" containerName="registry"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268415    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f3a5b22-bbac-4d45-b63d-98df62afb07a" containerName="console"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268421    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268425    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="kube-rbac-proxy-metric"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268432    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="prom-label-proxy"
Jul 21 00:23:01.268457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.268457    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" containerName="alertmanager"
Jul 21 00:23:01.269964 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.269945    2561 scope.go:117] "RemoveContainer" containerID="af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"
Jul 21 00:23:01.270255 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.270232    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422\": container with ID starting with af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422 not found: ID does not exist" containerID="af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"
Jul 21 00:23:01.270341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.270265    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"} err="failed to get container status \"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422\": rpc error: code = NotFound desc = could not find container \"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422\": container with ID starting with af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422 not found: ID does not exist"
Jul 21 00:23:01.270341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.270294    2561 scope.go:117] "RemoveContainer" containerID="ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"
Jul 21 00:23:01.270659 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.270640    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe\": container with ID starting with ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe not found: ID does not exist" containerID="ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"
Jul 21 00:23:01.270744 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.270664    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"} err="failed to get container status \"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe\": rpc error: code = NotFound desc = could not find container \"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe\": container with ID starting with ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe not found: ID does not exist"
Jul 21 00:23:01.270744 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.270687    2561 scope.go:117] "RemoveContainer" containerID="50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"
Jul 21 00:23:01.272677 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.272647    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4\": container with ID starting with 50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4 not found: ID does not exist" containerID="50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"
Jul 21 00:23:01.272770 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.272683    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"} err="failed to get container status \"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4\": rpc error: code = NotFound desc = could not find container \"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4\": container with ID starting with 50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4 not found: ID does not exist"
Jul 21 00:23:01.272770 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.272705    2561 scope.go:117] "RemoveContainer" containerID="1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"
Jul 21 00:23:01.272953 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.272933    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c\": container with ID starting with 1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c not found: ID does not exist" containerID="1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"
Jul 21 00:23:01.273001 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.272964    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"} err="failed to get container status \"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c\": rpc error: code = NotFound desc = could not find container \"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c\": container with ID starting with 1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c not found: ID does not exist"
Jul 21 00:23:01.273001 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.272985    2561 scope.go:117] "RemoveContainer" containerID="8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"
Jul 21 00:23:01.273252 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.273236    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305\": container with ID starting with 8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305 not found: ID does not exist" containerID="8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"
Jul 21 00:23:01.273301 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273256    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"} err="failed to get container status \"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305\": rpc error: code = NotFound desc = could not find container \"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305\": container with ID starting with 8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305 not found: ID does not exist"
Jul 21 00:23:01.273301 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273275    2561 scope.go:117] "RemoveContainer" containerID="544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"
Jul 21 00:23:01.273592 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.273574    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526\": container with ID starting with 544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526 not found: ID does not exist" containerID="544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"
Jul 21 00:23:01.273638 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273594    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.273638 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273596    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"} err="failed to get container status \"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526\": rpc error: code = NotFound desc = could not find container \"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526\": container with ID starting with 544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526 not found: ID does not exist"
Jul 21 00:23:01.273638 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273612    2561 scope.go:117] "RemoveContainer" containerID="603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"
Jul 21 00:23:01.273909 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:01.273834    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d\": container with ID starting with 603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d not found: ID does not exist" containerID="603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"
Jul 21 00:23:01.273909 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273863    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"} err="failed to get container status \"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d\": rpc error: code = NotFound desc = could not find container \"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d\": container with ID starting with 603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d not found: ID does not exist"
Jul 21 00:23:01.273909 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.273890    2561 scope.go:117] "RemoveContainer" containerID="af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"
Jul 21 00:23:01.274159 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274138    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422"} err="failed to get container status \"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422\": rpc error: code = NotFound desc = could not find container \"af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422\": container with ID starting with af423130186699fc6c82df10be36a1b6dc5a56eeaea94a535a8b46083ecb5422 not found: ID does not exist"
Jul 21 00:23:01.274215 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274163    2561 scope.go:117] "RemoveContainer" containerID="ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"
Jul 21 00:23:01.274412 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274394    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe"} err="failed to get container status \"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe\": rpc error: code = NotFound desc = could not find container \"ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe\": container with ID starting with ea3db8bd1c1047ea7f406c092cdbf40bbd5542dbbe5d36754ae0076c7d8d9bfe not found: ID does not exist"
Jul 21 00:23:01.274521 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274416    2561 scope.go:117] "RemoveContainer" containerID="50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"
Jul 21 00:23:01.274750 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274728    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4"} err="failed to get container status \"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4\": rpc error: code = NotFound desc = could not find container \"50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4\": container with ID starting with 50da3862c3b967550f0ff3a728e07a2fb83e69aa8e5cafbb7052f98e12d73fc4 not found: ID does not exist"
Jul 21 00:23:01.274802 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274754    2561 scope.go:117] "RemoveContainer" containerID="1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"
Jul 21 00:23:01.274999 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.274982    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c"} err="failed to get container status \"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c\": rpc error: code = NotFound desc = could not find container \"1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c\": container with ID starting with 1589ecf38e5b407157e10afaa00730c3c33e7ce627050e3e894e7ac9aa060c4c not found: ID does not exist"
Jul 21 00:23:01.275069 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275000    2561 scope.go:117] "RemoveContainer" containerID="8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"
Jul 21 00:23:01.275237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275214    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305"} err="failed to get container status \"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305\": rpc error: code = NotFound desc = could not find container \"8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305\": container with ID starting with 8b0b00ce433a2bb52352fce2240f21fe4430d922f8811efea72a5731871b4305 not found: ID does not exist"
Jul 21 00:23:01.275295 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275239    2561 scope.go:117] "RemoveContainer" containerID="544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"
Jul 21 00:23:01.275505 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275485    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526"} err="failed to get container status \"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526\": rpc error: code = NotFound desc = could not find container \"544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526\": container with ID starting with 544cd1e135506ab55f497f12e61f188717f123f44d68cb79a1236ada357b4526 not found: ID does not exist"
Jul 21 00:23:01.275577 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275506    2561 scope.go:117] "RemoveContainer" containerID="603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"
Jul 21 00:23:01.275759 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275737    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d"} err="failed to get container status \"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d\": rpc error: code = NotFound desc = could not find container \"603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d\": container with ID starting with 603d6a7c163f3f06527d4183103fd98c55b8978e6bd69531cf78360cdf51a00d not found: ID does not exist"
Jul 21 00:23:01.275832 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.275819    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 00:23:01.276122 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276085    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 00:23:01.276367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276185    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 00:23:01.276367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276210    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 00:23:01.276367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276215    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fmc2j\""
Jul 21 00:23:01.276367 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276188    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 00:23:01.276597 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276500    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 00:23:01.276597 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276572    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 00:23:01.276702 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.276596    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 00:23:01.282084 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.282067    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 00:23:01.286656 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.286637    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:23:01.291144 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291249 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291249 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291249 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291244    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j98mv\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-kube-api-access-j98mv\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291405 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291405 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291334    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291405 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291407    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291430    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-web-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291484    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291555    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-config-out\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.291641 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.291583    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.392888 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.392888 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392888    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392915    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-web-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392950    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.392977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393455 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393412    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-config-out\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393554 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393554 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393517    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393554 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393536    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393710 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.393710 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.393632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.394489 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.394428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j98mv\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-kube-api-access-j98mv\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.394597 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.394501    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.394797 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.394771    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.394979 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.394857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7ac892-bd4b-4868-925d-a1532bfeee53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-web-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396423 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7ac892-bd4b-4868-925d-a1532bfeee53-config-out\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396636 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396594    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396713 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396830 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.396876 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.396838    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.397316 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.397294    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.397737 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.397715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7ac892-bd4b-4868-925d-a1532bfeee53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.398146 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.398129    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.402131 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.402114    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j98mv\" (UniqueName: \"kubernetes.io/projected/da7ac892-bd4b-4868-925d-a1532bfeee53-kube-api-access-j98mv\") pod \"alertmanager-main-0\" (UID: \"da7ac892-bd4b-4868-925d-a1532bfeee53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.583423 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.583384    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 00:23:01.732197 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.732099    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 00:23:01.733997 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:23:01.733966    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7ac892_bd4b_4868_925d_a1532bfeee53.slice/crio-f12e4e69d18d4d0dfcdba20090088f741472dfb76fbc7ba5fe4d031a719f8bb7 WatchSource:0}: Error finding container f12e4e69d18d4d0dfcdba20090088f741472dfb76fbc7ba5fe4d031a719f8bb7: Status 404 returned error can't find the container with id f12e4e69d18d4d0dfcdba20090088f741472dfb76fbc7ba5fe4d031a719f8bb7
Jul 21 00:23:01.946916 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:01.946880    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5" path="/var/lib/kubelet/pods/4b1b1e9e-d0d9-48d6-9ad7-f47a9e2d95b5/volumes"
Jul 21 00:23:02.223555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:02.223472    2561 generic.go:356] "Generic (PLEG): container finished" podID="da7ac892-bd4b-4868-925d-a1532bfeee53" containerID="26f958e336e64da3d2ef8047c99b4fa9d196e371417f63d1515f50b74f2a6ac4" exitCode=0
Jul 21 00:23:02.223555 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:02.223517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerDied","Data":"26f958e336e64da3d2ef8047c99b4fa9d196e371417f63d1515f50b74f2a6ac4"}
Jul 21 00:23:02.223776 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:02.223567    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"f12e4e69d18d4d0dfcdba20090088f741472dfb76fbc7ba5fe4d031a719f8bb7"}
Jul 21 00:23:03.230504 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230468    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"23c4affffa6b057a0977a66a03dbf5dbe85dbed3db3fe4256ed9d1a04a3ef9a6"}
Jul 21 00:23:03.230928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230556    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"9018bdab35561f8f7c2e19e084e288f1137e267ba384dcfb2b49fa7645d74166"}
Jul 21 00:23:03.230928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230581    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"a656fa2a57f18cadcef2ac312f0ef51dfb9a69a3030a93ab365f3111e906c4f9"}
Jul 21 00:23:03.230928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230606    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"ef1a3c77f9c2d0afcb85f57c16a19ef0e7caae1bac3f0201f4eeac1f9d8f6039"}
Jul 21 00:23:03.230928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230629    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"714166773f730b3c2eeafe10b462cb095e03b717c53bd2638b1862516e13af37"}
Jul 21 00:23:03.230928 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:03.230652    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7ac892-bd4b-4868-925d-a1532bfeee53","Type":"ContainerStarted","Data":"b3b0ca16b80bcf68e973f4e7db448efe535f069443992103cc51e59beead35b2"}
Jul 21 00:23:12.630794 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:12.630737    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.615087306 podStartE2EDuration="11.630718278s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.630972ms" isStatefulPod=false podCreationTimestamp="2026-07-21 00:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:23:03.278023196 +0000 UTC m=+381.861886104" watchObservedRunningTime="2026-07-21 00:23:12.630718278 +0000 UTC m=+391.214581175"
Jul 21 00:23:12.631253 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:12.631220    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:23:37.652414 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:37.652370    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68bbddf9cc-5jhgw" podUID="1628e45b-f93f-470d-bd15-12ee8bf8199f" containerName="console" containerID="cri-o://94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce" gracePeriod=15
Jul 21 00:23:37.898573 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:37.898549    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:23:38.008843 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008746    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.008843 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008811    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.008843 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008829    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.008843 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008844    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.009166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008883    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcjbg\" (UniqueName: \"kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.009166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008925    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.009166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.008985    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config\") pod \"1628e45b-f93f-470d-bd15-12ee8bf8199f\" (UID: \"1628e45b-f93f-470d-bd15-12ee8bf8199f\") "
Jul 21 00:23:38.009411 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.009363    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:38.009730 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.009495    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:38.009730 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.009538    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca" (OuterVolumeSpecName: "service-ca") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:38.009849 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.009783    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config" (OuterVolumeSpecName: "console-config") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 00:23:38.011186 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.011165    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:38.011278 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.011180    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg" (OuterVolumeSpecName: "kube-api-access-gcjbg") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "kube-api-access-gcjbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:23:38.011278 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.011226    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1628e45b-f93f-470d-bd15-12ee8bf8199f" (UID: "1628e45b-f93f-470d-bd15-12ee8bf8199f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:23:38.110685 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110650    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110685 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110676    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110906 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110764    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1628e45b-f93f-470d-bd15-12ee8bf8199f-console-oauth-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110906 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110893    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-oauth-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110972 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110917    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-trusted-ca-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110972 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110926    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcjbg\" (UniqueName: \"kubernetes.io/projected/1628e45b-f93f-470d-bd15-12ee8bf8199f-kube-api-access-gcjbg\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.110972 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.110937    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1628e45b-f93f-470d-bd15-12ee8bf8199f-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:23:38.370081 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.370042    2561 generic.go:356] "Generic (PLEG): container finished" podID="1628e45b-f93f-470d-bd15-12ee8bf8199f" containerID="94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce" exitCode=0
Jul 21 00:23:38.370244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.370091    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bbddf9cc-5jhgw"
Jul 21 00:23:38.370244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.370124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bbddf9cc-5jhgw" event={"ID":"1628e45b-f93f-470d-bd15-12ee8bf8199f","Type":"ContainerDied","Data":"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"}
Jul 21 00:23:38.370244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.370175    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bbddf9cc-5jhgw" event={"ID":"1628e45b-f93f-470d-bd15-12ee8bf8199f","Type":"ContainerDied","Data":"9fdb031a56b07d7dfd6c8d8dde6b337194ab7998796ef5dccd2efa21f5af0329"}
Jul 21 00:23:38.370244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.370195    2561 scope.go:117] "RemoveContainer" containerID="94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"
Jul 21 00:23:38.378796 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.378772    2561 scope.go:117] "RemoveContainer" containerID="94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"
Jul 21 00:23:38.379090 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:23:38.379071    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce\": container with ID starting with 94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce not found: ID does not exist" containerID="94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"
Jul 21 00:23:38.379141 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.379100    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce"} err="failed to get container status \"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce\": rpc error: code = NotFound desc = could not find container \"94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce\": container with ID starting with 94f663beb62a3db62f4a517f680f7adc177defca65bf00b4224383bf342702ce not found: ID does not exist"
Jul 21 00:23:38.393079 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.393048    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:23:38.395637 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:38.395615    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68bbddf9cc-5jhgw"]
Jul 21 00:23:39.945384 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:23:39.945341    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1628e45b-f93f-470d-bd15-12ee8bf8199f" path="/var/lib/kubelet/pods/1628e45b-f93f-470d-bd15-12ee8bf8199f/volumes"
Jul 21 00:24:41.347506 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.347474    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lvpv"]
Jul 21 00:24:41.347954 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.347778    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1628e45b-f93f-470d-bd15-12ee8bf8199f" containerName="console"
Jul 21 00:24:41.347954 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.347788    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1628e45b-f93f-470d-bd15-12ee8bf8199f" containerName="console"
Jul 21 00:24:41.347954 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.347858    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1628e45b-f93f-470d-bd15-12ee8bf8199f" containerName="console"
Jul 21 00:24:41.350615 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.350599    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.353059 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.353034    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 00:24:41.353059 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.353049    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 00:24:41.353751 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.353738    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gtp4n\""
Jul 21 00:24:41.359882 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.359860    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lvpv"]
Jul 21 00:24:41.415529 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.415509    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.415636 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.415556    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkpzs\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-kube-api-access-jkpzs\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.516546 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.516522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.516663 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.516574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkpzs\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-kube-api-access-jkpzs\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.521900 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.521880    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"]
Jul 21 00:24:41.524935 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.524922    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.527075 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.527057    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kj2zm\""
Jul 21 00:24:41.528111 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.528085    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkpzs\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-kube-api-access-jkpzs\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.528415 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.528400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/117d2faf-8383-401a-a64d-45e2a1353c80-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lvpv\" (UID: \"117d2faf-8383-401a-a64d-45e2a1353c80\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.535298 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.535279    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"]
Jul 21 00:24:41.617011 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.616960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5hws\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-kube-api-access-c5hws\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.617011 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.616992    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.676140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.676117    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:41.718326 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.718293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.718471 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.718430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5hws\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-kube-api-access-c5hws\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.727605 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.727568    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.727729 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.727649    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5hws\" (UniqueName: \"kubernetes.io/projected/cb6297c2-0124-4a61-b43b-616d891809de-kube-api-access-c5hws\") pod \"cert-manager-cainjector-559cdd8c87-wmhr4\" (UID: \"cb6297c2-0124-4a61-b43b-616d891809de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.803792 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.803756    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lvpv"]
Jul 21 00:24:41.806873 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:24:41.806842    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod117d2faf_8383_401a_a64d_45e2a1353c80.slice/crio-b5e35c50159451543e4b15b319538037f59740a4df4e62de7deeb7db5e5c96cd WatchSource:0}: Error finding container b5e35c50159451543e4b15b319538037f59740a4df4e62de7deeb7db5e5c96cd: Status 404 returned error can't find the container with id b5e35c50159451543e4b15b319538037f59740a4df4e62de7deeb7db5e5c96cd
Jul 21 00:24:41.848321 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.848301    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kj2zm\""
Jul 21 00:24:41.856576 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.856555    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"
Jul 21 00:24:41.982296 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:41.982275    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4"]
Jul 21 00:24:41.984656 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:24:41.984628    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb6297c2_0124_4a61_b43b_616d891809de.slice/crio-80b430e2d4a8d5ee2143e9c3c05c13a9db9d74c5d40c0ffdd0a0f925a4d7d6df WatchSource:0}: Error finding container 80b430e2d4a8d5ee2143e9c3c05c13a9db9d74c5d40c0ffdd0a0f925a4d7d6df: Status 404 returned error can't find the container with id 80b430e2d4a8d5ee2143e9c3c05c13a9db9d74c5d40c0ffdd0a0f925a4d7d6df
Jul 21 00:24:42.626904 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:42.626868    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4" event={"ID":"cb6297c2-0124-4a61-b43b-616d891809de","Type":"ContainerStarted","Data":"80b430e2d4a8d5ee2143e9c3c05c13a9db9d74c5d40c0ffdd0a0f925a4d7d6df"}
Jul 21 00:24:42.628786 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:42.628723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv" event={"ID":"117d2faf-8383-401a-a64d-45e2a1353c80","Type":"ContainerStarted","Data":"b5e35c50159451543e4b15b319538037f59740a4df4e62de7deeb7db5e5c96cd"}
Jul 21 00:24:45.644834 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:45.644793    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4" event={"ID":"cb6297c2-0124-4a61-b43b-616d891809de","Type":"ContainerStarted","Data":"99060c5796568ef628c54732d678a19b25675f4249f709deb7f7858e6b720c35"}
Jul 21 00:24:45.646468 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:45.646432    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv" event={"ID":"117d2faf-8383-401a-a64d-45e2a1353c80","Type":"ContainerStarted","Data":"f63131974cf44aaaeaf15be6158aa780a5b3d25b491cb250721879d3fdb4e113"}
Jul 21 00:24:45.646585 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:45.646571    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:24:45.660913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:45.660873    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wmhr4" podStartSLOduration=1.397612746 podStartE2EDuration="4.66086006s" totalImagesPullingTime="3.263247314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:24:45.660358673 +0000 UTC m=+484.244221568" watchObservedRunningTime="2026-07-21 00:24:45.66086006 +0000 UTC m=+484.244722957"
Jul 21 00:24:45.676148 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:45.676097    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv" podStartSLOduration=1.245065924 podStartE2EDuration="4.676082663s" totalImagesPullingTime="3.431016739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:24:45.675300648 +0000 UTC m=+484.259163546" watchObservedRunningTime="2026-07-21 00:24:45.676082663 +0000 UTC m=+484.259945562"
Jul 21 00:24:51.653691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:24:51.653659    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7lvpv"
Jul 21 00:25:14.116066 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.116032    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"]
Jul 21 00:25:14.119392 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.119371    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.122775 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.122747    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 00:25:14.122930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.122816    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 00:25:14.122930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.122885    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5b66d\""
Jul 21 00:25:14.122930 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.122886    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 00:25:14.123131 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.123089    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 00:25:14.123131 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.123093    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 00:25:14.130585 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.130564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"]
Jul 21 00:25:14.177577 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.177542    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.177774 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.177595    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c17a1d09-6454-40c0-a6a8-49133610b017-manager-config\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.177774 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.177637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsbkv\" (UniqueName: \"kubernetes.io/projected/c17a1d09-6454-40c0-a6a8-49133610b017-kube-api-access-zsbkv\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.177774 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.177687    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-metrics-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.278595 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.278549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.278595 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.278602    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c17a1d09-6454-40c0-a6a8-49133610b017-manager-config\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.278811 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.278644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsbkv\" (UniqueName: \"kubernetes.io/projected/c17a1d09-6454-40c0-a6a8-49133610b017-kube-api-access-zsbkv\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.278811 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.278697    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-metrics-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.279336 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.279312    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c17a1d09-6454-40c0-a6a8-49133610b017-manager-config\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.281518 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.281493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.281612 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.281561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c17a1d09-6454-40c0-a6a8-49133610b017-metrics-cert\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.287294 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.287273    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsbkv\" (UniqueName: \"kubernetes.io/projected/c17a1d09-6454-40c0-a6a8-49133610b017-kube-api-access-zsbkv\") pod \"lws-controller-manager-75bcdcf5cd-qfvs8\" (UID: \"c17a1d09-6454-40c0-a6a8-49133610b017\") " pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.429986 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.429887    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:14.571875 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.571850    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"]
Jul 21 00:25:14.574635 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:25:14.574608    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17a1d09_6454_40c0_a6a8_49133610b017.slice/crio-6ca88a5fab693969eabb0e203a23d0488d54d035dbd94aa32db72c0cf56a15b9 WatchSource:0}: Error finding container 6ca88a5fab693969eabb0e203a23d0488d54d035dbd94aa32db72c0cf56a15b9: Status 404 returned error can't find the container with id 6ca88a5fab693969eabb0e203a23d0488d54d035dbd94aa32db72c0cf56a15b9
Jul 21 00:25:14.771875 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:14.771840    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8" event={"ID":"c17a1d09-6454-40c0-a6a8-49133610b017","Type":"ContainerStarted","Data":"6ca88a5fab693969eabb0e203a23d0488d54d035dbd94aa32db72c0cf56a15b9"}
Jul 21 00:25:17.788170 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:17.788132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8" event={"ID":"c17a1d09-6454-40c0-a6a8-49133610b017","Type":"ContainerStarted","Data":"4bec67ef23f0e36f3aff44ec3fc3a410ddf99d81dcf6df86c9631558e1312bf1"}
Jul 21 00:25:17.788691 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:17.788234    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:17.837719 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:17.837668    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8" podStartSLOduration=1.02792911 podStartE2EDuration="3.837647882s" totalImagesPullingTime="2.809718772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:25:17.836531907 +0000 UTC m=+516.420394835" watchObservedRunningTime="2026-07-21 00:25:17.837647882 +0000 UTC m=+516.421510778"
Jul 21 00:25:19.531979 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.531948    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"]
Jul 21 00:25:19.535915 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.535899    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.539554 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.539525    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 00:25:19.540244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.540215    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7h555\""
Jul 21 00:25:19.540350 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.540248    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 00:25:19.540411 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.540370    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 00:25:19.540575 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.540560    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 00:25:19.547953 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.547932    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"]
Jul 21 00:25:19.629190 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.629154    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.629363 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.629198    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-manifests\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.629363 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.629290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtl9s\" (UniqueName: \"kubernetes.io/projected/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-kube-api-access-qtl9s\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.629363 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.629344    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-tmp\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.629363 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.629363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.729892 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.729850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.730121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.729897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-manifests\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.730121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.729956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtl9s\" (UniqueName: \"kubernetes.io/projected/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-kube-api-access-qtl9s\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.730121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.730007    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-tmp\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.730121 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.730029    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.730331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.730305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-manifests\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.732498 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.732480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-tmp\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.732650 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.732634    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.732804 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.732784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.737687 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.737668    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtl9s\" (UniqueName: \"kubernetes.io/projected/d9ed9877-217d-4130-8e3b-4fd091f5e7cf-kube-api-access-qtl9s\") pod \"opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq\" (UID: \"d9ed9877-217d-4130-8e3b-4fd091f5e7cf\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.846420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.846328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:19.988395 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:19.988323    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"]
Jul 21 00:25:19.990966 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:25:19.990933    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ed9877_217d_4130_8e3b_4fd091f5e7cf.slice/crio-3456cf987d504e36a8b637cd51dd82cd373dd000cdcce03f8fa9648b0a44c82e WatchSource:0}: Error finding container 3456cf987d504e36a8b637cd51dd82cd373dd000cdcce03f8fa9648b0a44c82e: Status 404 returned error can't find the container with id 3456cf987d504e36a8b637cd51dd82cd373dd000cdcce03f8fa9648b0a44c82e
Jul 21 00:25:20.802457 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:20.802394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq" event={"ID":"d9ed9877-217d-4130-8e3b-4fd091f5e7cf","Type":"ContainerStarted","Data":"3456cf987d504e36a8b637cd51dd82cd373dd000cdcce03f8fa9648b0a44c82e"}
Jul 21 00:25:23.818255 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:23.818222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq" event={"ID":"d9ed9877-217d-4130-8e3b-4fd091f5e7cf","Type":"ContainerStarted","Data":"e98fd35bff1f513adf52a73b3b246faedd01df226dee055d5ad3b45307eef0a9"}
Jul 21 00:25:24.823816 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:24.823780    2561 generic.go:356] "Generic (PLEG): container finished" podID="d9ed9877-217d-4130-8e3b-4fd091f5e7cf" containerID="e98fd35bff1f513adf52a73b3b246faedd01df226dee055d5ad3b45307eef0a9" exitCode=0
Jul 21 00:25:24.824194 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:24.823867    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq" event={"ID":"d9ed9877-217d-4130-8e3b-4fd091f5e7cf","Type":"ContainerDied","Data":"e98fd35bff1f513adf52a73b3b246faedd01df226dee055d5ad3b45307eef0a9"}
Jul 21 00:25:25.829946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:25.829910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq" event={"ID":"d9ed9877-217d-4130-8e3b-4fd091f5e7cf","Type":"ContainerStarted","Data":"ccdd8917401984345a9a6f4af0ad7af5b60a7f112c4ce03eb8eae8b7755e375e"}
Jul 21 00:25:25.830423 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:25.830034    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:25.851619 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:25.851570    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq" podStartSLOduration=2.218165343 podStartE2EDuration="6.851552448s" totalImagesPullingTime="3.57236053s" totalInitContainerRuntime="1.061026575s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:25:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:25:25.849882749 +0000 UTC m=+524.433745667" watchObservedRunningTime="2026-07-21 00:25:25.851552448 +0000 UTC m=+524.435415345"
Jul 21 00:25:28.794272 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:28.794237    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75bcdcf5cd-qfvs8"
Jul 21 00:25:36.837149 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:36.837073    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq"
Jul 21 00:25:39.487039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.487001    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"]
Jul 21 00:25:39.490678 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.490655    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.493286 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.493256    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 00:25:39.494109 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.494085    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 00:25:39.494498 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.494479    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 00:25:39.495233 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.495194    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nnt5c\""
Jul 21 00:25:39.495416 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.495394    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 00:25:39.502701 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.502679    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"]
Jul 21 00:25:39.606943 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.606905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.607111 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.606956    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e4bcfb-1e63-4198-a845-83e6725c1acb-tmp\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.607111 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.607038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pv89\" (UniqueName: \"kubernetes.io/projected/16e4bcfb-1e63-4198-a845-83e6725c1acb-kube-api-access-7pv89\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.708355 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.708320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.708561 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.708361    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e4bcfb-1e63-4198-a845-83e6725c1acb-tmp\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.708561 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.708430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pv89\" (UniqueName: \"kubernetes.io/projected/16e4bcfb-1e63-4198-a845-83e6725c1acb-kube-api-access-7pv89\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.708659 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:25:39.708584    2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 21 00:25:39.708659 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:25:39.708657    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs podName:16e4bcfb-1e63-4198-a845-83e6725c1acb nodeName:}" failed. No retries permitted until 2026-07-21 00:25:40.208634463 +0000 UTC m=+538.792497338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs") pod "kube-auth-proxy-66766cb76c-wg8kx" (UID: "16e4bcfb-1e63-4198-a845-83e6725c1acb") : secret "kube-auth-proxy-tls" not found
Jul 21 00:25:39.710903 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.710876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e4bcfb-1e63-4198-a845-83e6725c1acb-tmp\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:39.724688 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:39.724658    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pv89\" (UniqueName: \"kubernetes.io/projected/16e4bcfb-1e63-4198-a845-83e6725c1acb-kube-api-access-7pv89\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:40.213959 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:40.213926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:40.216398 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:40.216369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e4bcfb-1e63-4198-a845-83e6725c1acb-tls-certs\") pod \"kube-auth-proxy-66766cb76c-wg8kx\" (UID: \"16e4bcfb-1e63-4198-a845-83e6725c1acb\") " pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:40.400824 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:40.400790    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"
Jul 21 00:25:40.535085 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:40.535063    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx"]
Jul 21 00:25:40.537278 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:25:40.537253    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e4bcfb_1e63_4198_a845_83e6725c1acb.slice/crio-10ca414626d2e715d91507490b030f0118d5e56e23c9a335434e4c8beac18cc3 WatchSource:0}: Error finding container 10ca414626d2e715d91507490b030f0118d5e56e23c9a335434e4c8beac18cc3: Status 404 returned error can't find the container with id 10ca414626d2e715d91507490b030f0118d5e56e23c9a335434e4c8beac18cc3
Jul 21 00:25:40.899738 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:40.899703    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx" event={"ID":"16e4bcfb-1e63-4198-a845-83e6725c1acb","Type":"ContainerStarted","Data":"10ca414626d2e715d91507490b030f0118d5e56e23c9a335434e4c8beac18cc3"}
Jul 21 00:25:43.645212 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:43.645172    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 00:25:43.915323 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:43.915233    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx" event={"ID":"16e4bcfb-1e63-4198-a845-83e6725c1acb","Type":"ContainerStarted","Data":"57f1108bcd67ce1bf07bc075478fd18debfede2f3d783a5eeb2dc24224aafc08"}
Jul 21 00:25:43.931327 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:25:43.931278    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66766cb76c-wg8kx" podStartSLOduration=1.828233702 podStartE2EDuration="4.931262677s" totalImagesPullingTime="3.103028975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:25:43.930862965 +0000 UTC m=+542.514725881" watchObservedRunningTime="2026-07-21 00:25:43.931262677 +0000 UTC m=+542.515125575"
Jul 21 00:26:41.893999 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:26:41.893965    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:26:41.895587 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:26:41.895565    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:27:23.856180 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.856144    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"]
Jul 21 00:27:23.859381 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.859358    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:23.862214 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.862190    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 21 00:27:23.862317 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.862223    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 00:27:23.862449 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.862423    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 21 00:27:23.863128 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.863106    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xfbv5\""
Jul 21 00:27:23.863237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.863130    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 00:27:23.870487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.870462    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"]
Jul 21 00:27:23.899099 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.899065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf22k\" (UniqueName: \"kubernetes.io/projected/bc91a25a-1bdb-4839-969d-121efe271220-kube-api-access-cf22k\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:23.899227 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.899109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:23.899227 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.899196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc91a25a-1bdb-4839-969d-121efe271220-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:23.999960 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.999925    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf22k\" (UniqueName: \"kubernetes.io/projected/bc91a25a-1bdb-4839-969d-121efe271220-kube-api-access-cf22k\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:23.999960 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.999962    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.000168 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:23.999994    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc91a25a-1bdb-4839-969d-121efe271220-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.000168 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:24.000087    2561 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 21 00:27:24.000168 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:24.000149    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert podName:bc91a25a-1bdb-4839-969d-121efe271220 nodeName:}" failed. No retries permitted until 2026-07-21 00:27:24.500133263 +0000 UTC m=+643.083996157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-q54j4" (UID: "bc91a25a-1bdb-4839-969d-121efe271220") : secret "plugin-serving-cert" not found
Jul 21 00:27:24.000654 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.000638    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc91a25a-1bdb-4839-969d-121efe271220-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.010594 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.010568    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf22k\" (UniqueName: \"kubernetes.io/projected/bc91a25a-1bdb-4839-969d-121efe271220-kube-api-access-cf22k\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.505390 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.505354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.507956 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.507932    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc91a25a-1bdb-4839-969d-121efe271220-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q54j4\" (UID: \"bc91a25a-1bdb-4839-969d-121efe271220\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.769309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.769283    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"
Jul 21 00:27:24.904115 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.904076    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4"]
Jul 21 00:27:24.906934 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:27:24.906896    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc91a25a_1bdb_4839_969d_121efe271220.slice/crio-647871b475768ed78a8bf4653ab606606c00362676198ad148ac4bb22a697bf8 WatchSource:0}: Error finding container 647871b475768ed78a8bf4653ab606606c00362676198ad148ac4bb22a697bf8: Status 404 returned error can't find the container with id 647871b475768ed78a8bf4653ab606606c00362676198ad148ac4bb22a697bf8
Jul 21 00:27:24.908156 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:24.908132    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 00:27:25.366731 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:25.366700    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4" event={"ID":"bc91a25a-1bdb-4839-969d-121efe271220","Type":"ContainerStarted","Data":"647871b475768ed78a8bf4653ab606606c00362676198ad148ac4bb22a697bf8"}
Jul 21 00:27:34.505862 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.505822    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"]
Jul 21 00:27:34.523039 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.522994    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.524976 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.524945    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"]
Jul 21 00:27:34.526362 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.526331    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-btcj9\""
Jul 21 00:27:34.595571 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.595538    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.595745 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.595602    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfjhv\" (UniqueName: \"kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.681932 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.681900    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"]
Jul 21 00:27:34.682235 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:34.682211    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-lfjhv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr" podUID="d233f0ee-b230-423a-afd5-067467a579d1"
Jul 21 00:27:34.691310 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.691280    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"]
Jul 21 00:27:34.696835 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.696805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.696995 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.696873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjhv\" (UniqueName: \"kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.697309 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.697279    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:34.699346 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:34.699327    2561 projected.go:196] Error preparing data for projected volume kube-api-access-lfjhv for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-140-105.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object
Jul 21 00:27:34.699478 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:34.699414    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv podName:d233f0ee-b230-423a-afd5-067467a579d1 nodeName:}" failed. No retries permitted until 2026-07-21 00:27:35.199397692 +0000 UTC m=+653.783260567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lfjhv" (UniqueName: "kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv") pod "kuadrant-operator-controller-manager-86878c769f-vrqfr" (UID: "d233f0ee-b230-423a-afd5-067467a579d1") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-140-105.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object
Jul 21 00:27:34.723492 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.723466    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:27:34.731084 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.731065    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.747147 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.747121    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:27:34.798078 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.798042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.798251 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.798125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcfl7\" (UniqueName: \"kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.899559 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.899523    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcfl7\" (UniqueName: \"kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.899715 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.899619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.900036 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.900013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:34.921852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:34.921824    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcfl7\" (UniqueName: \"kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbbnt\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:35.044460 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.044403    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:35.201573 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.201489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjhv\" (UniqueName: \"kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv\") pod \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:35.210011 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:35.209971    2561 projected.go:196] Error preparing data for projected volume kube-api-access-lfjhv for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-vrqfr" not found
Jul 21 00:27:35.210173 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:35.210067    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv podName:d233f0ee-b230-423a-afd5-067467a579d1 nodeName:}" failed. No retries permitted until 2026-07-21 00:27:36.210044965 +0000 UTC m=+654.793907860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfjhv" (UniqueName: "kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv") pod "kuadrant-operator-controller-manager-86878c769f-vrqfr" (UID: "d233f0ee-b230-423a-afd5-067467a579d1") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-vrqfr" not found
Jul 21 00:27:35.419602 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.419568    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:35.422097 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:35.422056    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="d233f0ee-b230-423a-afd5-067467a579d1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:35.429823 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.429790    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:35.432264 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:35.432232    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="d233f0ee-b230-423a-afd5-067467a579d1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:35.503627 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.503552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume\") pod \"d233f0ee-b230-423a-afd5-067467a579d1\" (UID: \"d233f0ee-b230-423a-afd5-067467a579d1\") "
Jul 21 00:27:35.503820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.503800    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfjhv\" (UniqueName: \"kubernetes.io/projected/d233f0ee-b230-423a-afd5-067467a579d1-kube-api-access-lfjhv\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:27:35.503896 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.503837    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d233f0ee-b230-423a-afd5-067467a579d1" (UID: "d233f0ee-b230-423a-afd5-067467a579d1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:27:35.604588 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.604558    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d233f0ee-b230-423a-afd5-067467a579d1-extensions-socket-volume\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:27:35.949264 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:35.947669    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d233f0ee-b230-423a-afd5-067467a579d1" path="/var/lib/kubelet/pods/d233f0ee-b230-423a-afd5-067467a579d1/volumes"
Jul 21 00:27:36.424299 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:36.424258    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:36.428808 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:27:36.428761    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-vrqfr\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="d233f0ee-b230-423a-afd5-067467a579d1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrqfr"
Jul 21 00:27:47.326729 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:47.326706    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:27:47.328928 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:27:47.328898    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a2cf1f_ed05_464b_8667_bd6c5cba9309.slice/crio-f753e668dbbaeb9a3c807e83a91dd2be93ac7b1ba24abc7b9e733fd6404d4852 WatchSource:0}: Error finding container f753e668dbbaeb9a3c807e83a91dd2be93ac7b1ba24abc7b9e733fd6404d4852: Status 404 returned error can't find the container with id f753e668dbbaeb9a3c807e83a91dd2be93ac7b1ba24abc7b9e733fd6404d4852
Jul 21 00:27:47.481628 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:47.481542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" event={"ID":"42a2cf1f-ed05-464b-8667-bd6c5cba9309","Type":"ContainerStarted","Data":"f753e668dbbaeb9a3c807e83a91dd2be93ac7b1ba24abc7b9e733fd6404d4852"}
Jul 21 00:27:48.487783 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:48.487750    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4" event={"ID":"bc91a25a-1bdb-4839-969d-121efe271220","Type":"ContainerStarted","Data":"52a35864ef6d3a8910440dd037d2210c9aa64f8523902c362708c343c1723fcb"}
Jul 21 00:27:48.504140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:48.504090    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q54j4" podStartSLOduration=2.8388360500000003 podStartE2EDuration="25.504074454s" totalImagesPullingTime="22.665238404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:27:48.502022703 +0000 UTC m=+667.085885602" watchObservedRunningTime="2026-07-21 00:27:48.504074454 +0000 UTC m=+667.087937351"
Jul 21 00:27:56.531459 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:56.531400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" event={"ID":"42a2cf1f-ed05-464b-8667-bd6c5cba9309","Type":"ContainerStarted","Data":"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"}
Jul 21 00:27:56.531977 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:56.531514    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:27:56.555488 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:27:56.555418    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" podStartSLOduration=14.384891183 podStartE2EDuration="22.555401051s" totalImagesPullingTime="8.170509868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:27:56.553351286 +0000 UTC m=+675.137214208" watchObservedRunningTime="2026-07-21 00:27:56.555401051 +0000 UTC m=+675.139263947"
Jul 21 00:28:07.538547 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:07.538513    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:28:27.157942 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.157905    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:27.236501 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.236469    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:27.236686 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.236464    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:27.238982 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.238959    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zx4g5\""
Jul 21 00:28:27.367567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.367538    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txz87\" (UniqueName: \"kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87\") pod \"authorino-75d556bd77-2rskl\" (UID: \"dae9860e-0296-46f3-b23c-44703e6e1c14\") " pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:27.468041 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.467951    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txz87\" (UniqueName: \"kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87\") pod \"authorino-75d556bd77-2rskl\" (UID: \"dae9860e-0296-46f3-b23c-44703e6e1c14\") " pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:27.477458 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.477407    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txz87\" (UniqueName: \"kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87\") pod \"authorino-75d556bd77-2rskl\" (UID: \"dae9860e-0296-46f3-b23c-44703e6e1c14\") " pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:27.545151 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.545120    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:27.679112 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:27.679089    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:27.681244 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:28:27.681212    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddae9860e_0296_46f3_b23c_44703e6e1c14.slice/crio-68e21ed61ab0586d950fd6773d7ab48077efb80e3ce38fe710e041a8fede47b8 WatchSource:0}: Error finding container 68e21ed61ab0586d950fd6773d7ab48077efb80e3ce38fe710e041a8fede47b8: Status 404 returned error can't find the container with id 68e21ed61ab0586d950fd6773d7ab48077efb80e3ce38fe710e041a8fede47b8
Jul 21 00:28:28.678815 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:28.678760    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2rskl" event={"ID":"dae9860e-0296-46f3-b23c-44703e6e1c14","Type":"ContainerStarted","Data":"68e21ed61ab0586d950fd6773d7ab48077efb80e3ce38fe710e041a8fede47b8"}
Jul 21 00:28:31.693820 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:31.693781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2rskl" event={"ID":"dae9860e-0296-46f3-b23c-44703e6e1c14","Type":"ContainerStarted","Data":"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"}
Jul 21 00:28:31.710172 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:31.710122    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-2rskl" podStartSLOduration=1.321445108 podStartE2EDuration="4.710105615s" totalImagesPullingTime="3.388660507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:28:31.708187175 +0000 UTC m=+710.292050071" watchObservedRunningTime="2026-07-21 00:28:31.710105615 +0000 UTC m=+710.293968511"
Jul 21 00:28:56.378053 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.378015    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:56.378591 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.378237    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-2rskl" podUID="dae9860e-0296-46f3-b23c-44703e6e1c14" containerName="authorino" containerID="cri-o://208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199" gracePeriod=30
Jul 21 00:28:56.622399 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.622371    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:56.730022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.729936    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txz87\" (UniqueName: \"kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87\") pod \"dae9860e-0296-46f3-b23c-44703e6e1c14\" (UID: \"dae9860e-0296-46f3-b23c-44703e6e1c14\") "
Jul 21 00:28:56.732251 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.732225    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87" (OuterVolumeSpecName: "kube-api-access-txz87") pod "dae9860e-0296-46f3-b23c-44703e6e1c14" (UID: "dae9860e-0296-46f3-b23c-44703e6e1c14"). InnerVolumeSpecName "kube-api-access-txz87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:28:56.807755 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.807717    2561 generic.go:356] "Generic (PLEG): container finished" podID="dae9860e-0296-46f3-b23c-44703e6e1c14" containerID="208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199" exitCode=0
Jul 21 00:28:56.807946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.807765    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2rskl"
Jul 21 00:28:56.807946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.807799    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2rskl" event={"ID":"dae9860e-0296-46f3-b23c-44703e6e1c14","Type":"ContainerDied","Data":"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"}
Jul 21 00:28:56.807946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.807838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2rskl" event={"ID":"dae9860e-0296-46f3-b23c-44703e6e1c14","Type":"ContainerDied","Data":"68e21ed61ab0586d950fd6773d7ab48077efb80e3ce38fe710e041a8fede47b8"}
Jul 21 00:28:56.807946 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.807859    2561 scope.go:117] "RemoveContainer" containerID="208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"
Jul 21 00:28:56.817289 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.817272    2561 scope.go:117] "RemoveContainer" containerID="208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"
Jul 21 00:28:56.817590 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:28:56.817570    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199\": container with ID starting with 208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199 not found: ID does not exist" containerID="208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"
Jul 21 00:28:56.817665 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.817599    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199"} err="failed to get container status \"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199\": rpc error: code = NotFound desc = could not find container \"208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199\": container with ID starting with 208062eeb519dbf506509cc96451ecbbb33111eeebc382fdc61311ee57f87199 not found: ID does not exist"
Jul 21 00:28:56.827803 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.827777    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:56.830826 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.830799    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txz87\" (UniqueName: \"kubernetes.io/projected/dae9860e-0296-46f3-b23c-44703e6e1c14-kube-api-access-txz87\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:28:56.831386 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:56.831365    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2rskl"]
Jul 21 00:28:57.947640 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:28:57.947603    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dae9860e-0296-46f3-b23c-44703e6e1c14" path="/var/lib/kubelet/pods/dae9860e-0296-46f3-b23c-44703e6e1c14/volumes"
Jul 21 00:29:01.503237 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.503204    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:29:01.503822 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.503714    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae9860e-0296-46f3-b23c-44703e6e1c14" containerName="authorino"
Jul 21 00:29:01.503822 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.503733    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae9860e-0296-46f3-b23c-44703e6e1c14" containerName="authorino"
Jul 21 00:29:01.503934 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.503841    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dae9860e-0296-46f3-b23c-44703e6e1c14" containerName="authorino"
Jul 21 00:29:01.517597 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.517566    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.520507 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.520482    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 00:29:01.520707 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.520686    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nhqrc\""
Jul 21 00:29:01.520832 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.520549    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 00:29:01.521956 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.521936    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:29:01.575300 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.575265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7j9\" (UniqueName: \"kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.575480 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.575313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.676166 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.676134    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7j9\" (UniqueName: \"kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.676331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.676193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.678873 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.678851    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.686387 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.686359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7j9\" (UniqueName: \"kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9\") pod \"maas-controller-79bfd5d9f6-lpb82\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") " pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.829558 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.829529    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:01.991736 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:01.991716    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:29:01.994100 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:29:01.994075    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd0f770_adf7_47e5_9693_668e92a9dc8d.slice/crio-f31b65204a9818eecfa73085b1b45ae85e52c356844737eaa8c7c0125e8948c6 WatchSource:0}: Error finding container f31b65204a9818eecfa73085b1b45ae85e52c356844737eaa8c7c0125e8948c6: Status 404 returned error can't find the container with id f31b65204a9818eecfa73085b1b45ae85e52c356844737eaa8c7c0125e8948c6
Jul 21 00:29:02.839013 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:02.838970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" event={"ID":"6fd0f770-adf7-47e5-9693-668e92a9dc8d","Type":"ContainerStarted","Data":"f31b65204a9818eecfa73085b1b45ae85e52c356844737eaa8c7c0125e8948c6"}
Jul 21 00:29:04.849240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:04.849210    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" event={"ID":"6fd0f770-adf7-47e5-9693-668e92a9dc8d","Type":"ContainerStarted","Data":"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"}
Jul 21 00:29:04.849613 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:04.849271    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:15.857114 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:15.857088    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:29:15.877275 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:15.877223    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" podStartSLOduration=12.662549817 podStartE2EDuration="14.877206647s" totalImagesPullingTime="2.21465683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:29:04.868259015 +0000 UTC m=+743.452121912" watchObservedRunningTime="2026-07-21 00:29:15.877206647 +0000 UTC m=+754.461069543"
Jul 21 00:29:36.157855 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.157819    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"]
Jul 21 00:29:36.160049 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.160034    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.163133 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.163112    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 00:29:36.177112 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.177084    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"]
Jul 21 00:29:36.183003 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.182974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx55b\" (UniqueName: \"kubernetes.io/projected/20d1ece1-ab82-4ab1-b3ce-804b2612e620-kube-api-access-kx55b\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.183130 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.183037    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1ece1-ab82-4ab1-b3ce-804b2612e620-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.284160 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.284122    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1ece1-ab82-4ab1-b3ce-804b2612e620-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.284341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.284229    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx55b\" (UniqueName: \"kubernetes.io/projected/20d1ece1-ab82-4ab1-b3ce-804b2612e620-kube-api-access-kx55b\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.285088 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.285059    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1ece1-ab82-4ab1-b3ce-804b2612e620-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.304164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.304137    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx55b\" (UniqueName: \"kubernetes.io/projected/20d1ece1-ab82-4ab1-b3ce-804b2612e620-kube-api-access-kx55b\") pod \"payload-pre-processing-7677b774bb-jh8tg\" (UID: \"20d1ece1-ab82-4ab1-b3ce-804b2612e620\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.470127 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.470037    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:36.612784 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.612758    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"]
Jul 21 00:29:36.614927 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:29:36.614903    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d1ece1_ab82_4ab1_b3ce_804b2612e620.slice/crio-11909ff1290f0c60af45926438d41ef52c01fae807cf2af81afcc23d3670c1b1 WatchSource:0}: Error finding container 11909ff1290f0c60af45926438d41ef52c01fae807cf2af81afcc23d3670c1b1: Status 404 returned error can't find the container with id 11909ff1290f0c60af45926438d41ef52c01fae807cf2af81afcc23d3670c1b1
Jul 21 00:29:36.996590 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:36.996557    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg" event={"ID":"20d1ece1-ab82-4ab1-b3ce-804b2612e620","Type":"ContainerStarted","Data":"11909ff1290f0c60af45926438d41ef52c01fae807cf2af81afcc23d3670c1b1"}
Jul 21 00:29:39.008465 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:39.008405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg" event={"ID":"20d1ece1-ab82-4ab1-b3ce-804b2612e620","Type":"ContainerStarted","Data":"70356446b8bfdaf8ac7d3bf7ce68bbd492b78cda849d87c2f28cf0bb1e4a6414"}
Jul 21 00:29:39.008949 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:39.008499    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:39.028234 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:39.028176    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg" podStartSLOduration=1.509521399 podStartE2EDuration="3.028159394s" totalImagesPullingTime="1.518637995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:29:39.026737979 +0000 UTC m=+777.610600896" watchObservedRunningTime="2026-07-21 00:29:39.028159394 +0000 UTC m=+777.612022291"
Jul 21 00:29:50.017628 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:50.017597    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jh8tg"
Jul 21 00:29:57.070480 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.070426    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"]
Jul 21 00:29:57.075646 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.075619    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.079190 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.079169    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m5dwj\""
Jul 21 00:29:57.080187 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.080166    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 21 00:29:57.080288 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.080170    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 00:29:57.080288 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.080201    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 00:29:57.086829 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.086807    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"]
Jul 21 00:29:57.168430 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.168616 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e96d50bd-0176-4d20-b17a-8218f709db39-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.168616 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p79vf\" (UniqueName: \"kubernetes.io/projected/e96d50bd-0176-4d20-b17a-8218f709db39-kube-api-access-p79vf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.168616 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168607    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.168739 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168632    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.168739 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.168665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.269990    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270022 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e96d50bd-0176-4d20-b17a-8218f709db39-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270262 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270042    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p79vf\" (UniqueName: \"kubernetes.io/projected/e96d50bd-0176-4d20-b17a-8218f709db39-kube-api-access-p79vf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270262 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270262 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270262 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270148    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270591 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270591 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270586    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.270713 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.270614    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.272421 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.272398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e96d50bd-0176-4d20-b17a-8218f709db39-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.272718 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.272701    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e96d50bd-0176-4d20-b17a-8218f709db39-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.278889 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.278869    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p79vf\" (UniqueName: \"kubernetes.io/projected/e96d50bd-0176-4d20-b17a-8218f709db39-kube-api-access-p79vf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh\" (UID: \"e96d50bd-0176-4d20-b17a-8218f709db39\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.386629 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.386565    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:29:57.524580 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:57.524558    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"]
Jul 21 00:29:57.526883 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:29:57.526849    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96d50bd_0176_4d20_b17a_8218f709db39.slice/crio-d25611c45b95b814c5bb292b15bd456751713d7359cb64d1cfb10e17eaad14d2 WatchSource:0}: Error finding container d25611c45b95b814c5bb292b15bd456751713d7359cb64d1cfb10e17eaad14d2: Status 404 returned error can't find the container with id d25611c45b95b814c5bb292b15bd456751713d7359cb64d1cfb10e17eaad14d2
Jul 21 00:29:58.101920 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:58.101865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh" event={"ID":"e96d50bd-0176-4d20-b17a-8218f709db39","Type":"ContainerStarted","Data":"d25611c45b95b814c5bb292b15bd456751713d7359cb64d1cfb10e17eaad14d2"}
Jul 21 00:29:59.107623 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:59.107529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh" event={"ID":"e96d50bd-0176-4d20-b17a-8218f709db39","Type":"ContainerStarted","Data":"ebb2adc236e6d198be535bd768d028375c614769402f29a93845053c7b2519fe"}
Jul 21 00:29:59.127990 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:29:59.127937    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh" podStartSLOduration=0.872697312 podStartE2EDuration="2.127922478s" totalImagesPullingTime="1.255225166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:29:59.125638003 +0000 UTC m=+797.709500900" watchObservedRunningTime="2026-07-21 00:29:59.127922478 +0000 UTC m=+797.711785374"
Jul 21 00:30:07.386945 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:07.386906    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:30:07.387355 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:07.387047    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:30:07.399806 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:07.399781    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:30:08.162014 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:08.161984    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh"
Jul 21 00:30:46.844325 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.844290    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"]
Jul 21 00:30:46.847350 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.847324    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.849755 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.849733    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 00:30:46.858257 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.858237    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"]
Jul 21 00:30:46.894753 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894719    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5cfs\" (UniqueName: \"kubernetes.io/projected/626db352-8a4a-4041-87f4-b845f33f67b7-kube-api-access-t5cfs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.894910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894764    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.894910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894808    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.894910 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.895029 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.895029 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.894942    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/626db352-8a4a-4041-87f4-b845f33f67b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996307 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5cfs\" (UniqueName: \"kubernetes.io/projected/626db352-8a4a-4041-87f4-b845f33f67b7-kube-api-access-t5cfs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996478 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996478 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996671 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/626db352-8a4a-4041-87f4-b845f33f67b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996819 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996797    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996900 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996863    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.996955 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.996925    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.998692 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.998676    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/626db352-8a4a-4041-87f4-b845f33f67b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:46.999129 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:46.999106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/626db352-8a4a-4041-87f4-b845f33f67b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:47.004664 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:47.004644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5cfs\" (UniqueName: \"kubernetes.io/projected/626db352-8a4a-4041-87f4-b845f33f67b7-kube-api-access-t5cfs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tl978\" (UID: \"626db352-8a4a-4041-87f4-b845f33f67b7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:47.158497 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:47.158389    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:47.294385 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:47.294359    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"]
Jul 21 00:30:47.296902 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:30:47.296878    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626db352_8a4a_4041_87f4_b845f33f67b7.slice/crio-2115eba6951ead309683a0f095404cdcaa4879f7935d9c2dc67c448d5b9edecb WatchSource:0}: Error finding container 2115eba6951ead309683a0f095404cdcaa4879f7935d9c2dc67c448d5b9edecb: Status 404 returned error can't find the container with id 2115eba6951ead309683a0f095404cdcaa4879f7935d9c2dc67c448d5b9edecb
Jul 21 00:30:47.337168 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:47.337140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978" event={"ID":"626db352-8a4a-4041-87f4-b845f33f67b7","Type":"ContainerStarted","Data":"2115eba6951ead309683a0f095404cdcaa4879f7935d9c2dc67c448d5b9edecb"}
Jul 21 00:30:48.344023 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:48.343991    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978" event={"ID":"626db352-8a4a-4041-87f4-b845f33f67b7","Type":"ContainerStarted","Data":"851484f349ab03928abe60370ba8aef00b1bcf72d8da4d2a9072e25e5e944aca"}
Jul 21 00:30:48.363942 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:48.363890    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978" podStartSLOduration=2.158306048 podStartE2EDuration="2.363873324s" totalImagesPullingTime="205.567276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:30:48.362700829 +0000 UTC m=+846.946563733" watchObservedRunningTime="2026-07-21 00:30:48.363873324 +0000 UTC m=+846.947736221"
Jul 21 00:30:57.159217 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:57.159179    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:57.159676 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:57.159243    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:57.171791 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:57.171763    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:30:57.400687 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:30:57.400657    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tl978"
Jul 21 00:31:41.922207 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:31:41.922132    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:31:41.925188 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:31:41.925162    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:32:05.171788 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.171750    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cd88c9d98-52srp"]
Jul 21 00:32:05.173834 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.173818    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.177674 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.177655    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zx4g5\""
Jul 21 00:32:05.177674 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.177666    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 00:32:05.190217 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.190193    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cd88c9d98-52srp"]
Jul 21 00:32:05.313802 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.313772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-tls-cert\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.314009 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.313832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgsnp\" (UniqueName: \"kubernetes.io/projected/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-kube-api-access-fgsnp\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.414365 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.414328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgsnp\" (UniqueName: \"kubernetes.io/projected/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-kube-api-access-fgsnp\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.414593 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.414494    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-tls-cert\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.417126 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.417101    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-tls-cert\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.423758 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.423702    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgsnp\" (UniqueName: \"kubernetes.io/projected/4b012b1f-5ada-41f4-a0ad-44bc5f3ae073-kube-api-access-fgsnp\") pod \"authorino-6cd88c9d98-52srp\" (UID: \"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073\") " pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.483715 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.483665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cd88c9d98-52srp"
Jul 21 00:32:05.648056 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.648032    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cd88c9d98-52srp"]
Jul 21 00:32:05.650382 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:32:05.650355    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b012b1f_5ada_41f4_a0ad_44bc5f3ae073.slice/crio-52af325c6270618a67ceeeff6d1cb30a3d84dd176387add991d7efb2c061b43d WatchSource:0}: Error finding container 52af325c6270618a67ceeeff6d1cb30a3d84dd176387add991d7efb2c061b43d: Status 404 returned error can't find the container with id 52af325c6270618a67ceeeff6d1cb30a3d84dd176387add991d7efb2c061b43d
Jul 21 00:32:05.716323 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:05.716234    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd88c9d98-52srp" event={"ID":"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073","Type":"ContainerStarted","Data":"52af325c6270618a67ceeeff6d1cb30a3d84dd176387add991d7efb2c061b43d"}
Jul 21 00:32:06.724140 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:06.724096    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cd88c9d98-52srp" event={"ID":"4b012b1f-5ada-41f4-a0ad-44bc5f3ae073","Type":"ContainerStarted","Data":"6c1ae39d64e6e0b8a9e4bac2ee879102a67c8b29a14043a09206c9c9b7441593"}
Jul 21 00:32:53.340487 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.340415    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cd88c9d98-52srp" podStartSLOduration=47.736795567 podStartE2EDuration="48.340396132s" totalImagesPullingTime="603.600565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:32:06.759313821 +0000 UTC m=+925.343176724" watchObservedRunningTime="2026-07-21 00:32:53.340396132 +0000 UTC m=+971.924259028"
Jul 21 00:32:53.341174 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.341157    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:32:53.341375 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.341355    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" podUID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" containerName="manager" containerID="cri-o://fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e" gracePeriod=30
Jul 21 00:32:53.591134 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.591078    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:32:53.662232 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.662197    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert\") pod \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") "
Jul 21 00:32:53.662383 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.662291    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk7j9\" (UniqueName: \"kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9\") pod \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\" (UID: \"6fd0f770-adf7-47e5-9693-668e92a9dc8d\") "
Jul 21 00:32:53.664408 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.664379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6fd0f770-adf7-47e5-9693-668e92a9dc8d" (UID: "6fd0f770-adf7-47e5-9693-668e92a9dc8d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:32:53.664543 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.664407    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9" (OuterVolumeSpecName: "kube-api-access-wk7j9") pod "6fd0f770-adf7-47e5-9693-668e92a9dc8d" (UID: "6fd0f770-adf7-47e5-9693-668e92a9dc8d"). InnerVolumeSpecName "kube-api-access-wk7j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:32:53.763776 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.763740    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6fd0f770-adf7-47e5-9693-668e92a9dc8d-webhook-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:32:53.763776 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.763769    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk7j9\" (UniqueName: \"kubernetes.io/projected/6fd0f770-adf7-47e5-9693-668e92a9dc8d-kube-api-access-wk7j9\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:32:53.950913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.950830    2561 generic.go:356] "Generic (PLEG): container finished" podID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" containerID="fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e" exitCode=0
Jul 21 00:32:53.950913 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.950876    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82"
Jul 21 00:32:53.951147 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.950918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" event={"ID":"6fd0f770-adf7-47e5-9693-668e92a9dc8d","Type":"ContainerDied","Data":"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"}
Jul 21 00:32:53.951147 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.950970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bfd5d9f6-lpb82" event={"ID":"6fd0f770-adf7-47e5-9693-668e92a9dc8d","Type":"ContainerDied","Data":"f31b65204a9818eecfa73085b1b45ae85e52c356844737eaa8c7c0125e8948c6"}
Jul 21 00:32:53.951147 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.950998    2561 scope.go:117] "RemoveContainer" containerID="fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"
Jul 21 00:32:53.961390 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.961371    2561 scope.go:117] "RemoveContainer" containerID="fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"
Jul 21 00:32:53.961707 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:32:53.961688    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e\": container with ID starting with fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e not found: ID does not exist" containerID="fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"
Jul 21 00:32:53.961774 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.961716    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e"} err="failed to get container status \"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e\": rpc error: code = NotFound desc = could not find container \"fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e\": container with ID starting with fbe15f5e83480c181c0d50b5711567e64d098e5da7c6d8e705975f689e94e55e not found: ID does not exist"
Jul 21 00:32:53.971756 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.971732    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:32:53.975468 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:53.975427    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79bfd5d9f6-lpb82"]
Jul 21 00:32:55.948164 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:32:55.948130    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" path="/var/lib/kubelet/pods/6fd0f770-adf7-47e5-9693-668e92a9dc8d/volumes"
Jul 21 00:36:41.952683 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:36:41.952605    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:36:41.958132 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:36:41.958097    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:38:56.915775 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:38:56.915735    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:38:56.918397 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:38:56.915968    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" podUID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" containerName="manager" containerID="cri-o://a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf" gracePeriod=10
Jul 21 00:39:00.181935 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.181913    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:39:00.249295 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.249193    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume\") pod \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") "
Jul 21 00:39:00.249295 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.249254    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcfl7\" (UniqueName: \"kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7\") pod \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\" (UID: \"42a2cf1f-ed05-464b-8667-bd6c5cba9309\") "
Jul 21 00:39:00.249697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.249667    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "42a2cf1f-ed05-464b-8667-bd6c5cba9309" (UID: "42a2cf1f-ed05-464b-8667-bd6c5cba9309"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:39:00.251645 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.251616    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7" (OuterVolumeSpecName: "kube-api-access-vcfl7") pod "42a2cf1f-ed05-464b-8667-bd6c5cba9309" (UID: "42a2cf1f-ed05-464b-8667-bd6c5cba9309"). InnerVolumeSpecName "kube-api-access-vcfl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:39:00.350244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.350205    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42a2cf1f-ed05-464b-8667-bd6c5cba9309-extensions-socket-volume\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:39:00.350244 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.350237    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcfl7\" (UniqueName: \"kubernetes.io/projected/42a2cf1f-ed05-464b-8667-bd6c5cba9309-kube-api-access-vcfl7\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:39:00.736167 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.736137    2561 generic.go:356] "Generic (PLEG): container finished" podID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" containerID="a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf" exitCode=0
Jul 21 00:39:00.736344 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.736182    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"
Jul 21 00:39:00.736344 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.736205    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" event={"ID":"42a2cf1f-ed05-464b-8667-bd6c5cba9309","Type":"ContainerDied","Data":"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"}
Jul 21 00:39:00.736344 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.736248    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt" event={"ID":"42a2cf1f-ed05-464b-8667-bd6c5cba9309","Type":"ContainerDied","Data":"f753e668dbbaeb9a3c807e83a91dd2be93ac7b1ba24abc7b9e733fd6404d4852"}
Jul 21 00:39:00.736344 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.736268    2561 scope.go:117] "RemoveContainer" containerID="a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"
Jul 21 00:39:00.746599 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.746582    2561 scope.go:117] "RemoveContainer" containerID="a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"
Jul 21 00:39:00.746870 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:39:00.746851    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf\": container with ID starting with a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf not found: ID does not exist" containerID="a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"
Jul 21 00:39:00.746914 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.746878    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf"} err="failed to get container status \"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf\": rpc error: code = NotFound desc = could not find container \"a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf\": container with ID starting with a51fd297979f47fd0426e62cfcbd5ab23ab2775a7ac1b89f6d4c3194d6cc57bf not found: ID does not exist"
Jul 21 00:39:00.757647 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.757623    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:39:00.760842 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:00.760819    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbbnt"]
Jul 21 00:39:01.951717 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:01.951682    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" path="/var/lib/kubelet/pods/42a2cf1f-ed05-464b-8667-bd6c5cba9309/volumes"
Jul 21 00:39:58.128494 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.128452    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"]
Jul 21 00:39:58.129103 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.128968    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" containerName="manager"
Jul 21 00:39:58.129103 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.128987    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" containerName="manager"
Jul 21 00:39:58.129103 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.129033    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" containerName="manager"
Jul 21 00:39:58.129103 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.129043    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" containerName="manager"
Jul 21 00:39:58.129306 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.129122    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd0f770-adf7-47e5-9693-668e92a9dc8d" containerName="manager"
Jul 21 00:39:58.129306 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.129137    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a2cf1f-ed05-464b-8667-bd6c5cba9309" containerName="manager"
Jul 21 00:39:58.133504 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.133484    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.137754 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.137732    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-btcj9\""
Jul 21 00:39:58.147686 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.147663    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"]
Jul 21 00:39:58.254958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.254912    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.254958 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.254967    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgcd7\" (UniqueName: \"kubernetes.io/projected/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-kube-api-access-rgcd7\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.356416 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.356373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.356416 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.356426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgcd7\" (UniqueName: \"kubernetes.io/projected/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-kube-api-access-rgcd7\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.356793 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.356775    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.368923 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.368894    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgcd7\" (UniqueName: \"kubernetes.io/projected/e1cd4ba4-a496-4e61-b926-3bf167b70e6d-kube-api-access-rgcd7\") pod \"kuadrant-operator-controller-manager-86878c769f-r7b65\" (UID: \"e1cd4ba4-a496-4e61-b926-3bf167b70e6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.443933 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.443861    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:58.582772 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:58.582747    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"]
Jul 21 00:39:58.585806 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:39:58.585781    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1cd4ba4_a496_4e61_b926_3bf167b70e6d.slice/crio-b423d2744f44648f2e9203f7ab864b3d4b1b75b5fb37956a982cd2db5c47667b WatchSource:0}: Error finding container b423d2744f44648f2e9203f7ab864b3d4b1b75b5fb37956a982cd2db5c47667b: Status 404 returned error can't find the container with id b423d2744f44648f2e9203f7ab864b3d4b1b75b5fb37956a982cd2db5c47667b
Jul 21 00:39:59.005389 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:59.005350    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65" event={"ID":"e1cd4ba4-a496-4e61-b926-3bf167b70e6d","Type":"ContainerStarted","Data":"1e1da0520a4aa41261d2776ae88074e6cbbaf70a867e488e064b15007aa64bdf"}
Jul 21 00:39:59.005592 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:59.005408    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65" event={"ID":"e1cd4ba4-a496-4e61-b926-3bf167b70e6d","Type":"ContainerStarted","Data":"b423d2744f44648f2e9203f7ab864b3d4b1b75b5fb37956a982cd2db5c47667b"}
Jul 21 00:39:59.005592 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:59.005461    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:39:59.029960 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:39:59.029914    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65" podStartSLOduration=1.029898954 podStartE2EDuration="1.029898954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:39:59.028268246 +0000 UTC m=+1397.612131144" watchObservedRunningTime="2026-07-21 00:39:59.029898954 +0000 UTC m=+1397.613761850"
Jul 21 00:40:10.012656 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:40:10.012616    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r7b65"
Jul 21 00:41:41.981991 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:41:41.981951    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:41:41.988644 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:41:41.988625    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:42:49.137291 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.137255    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:42:49.151329 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.151292    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:42:49.151529 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.151480    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.154768 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.154742    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e58bdff0-kserve-self-signed-certs\""
Jul 21 00:42:49.213068 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.213246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213080    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnv5\" (UniqueName: \"kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.213246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.213246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.213246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.213246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.213229    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314371 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314379    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314414    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314467    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314673 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314607    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnv5\" (UniqueName: \"kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314884 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314850    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.314974 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.314954    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.315034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.315007    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.316926 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.316897    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.317165 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.317150    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.322513 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.322492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnv5\" (UniqueName: \"kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5\") pod \"e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") " pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.461881 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.461778    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:49.596379 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.596353    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:42:49.598961 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:42:49.598935    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f30bb7c_a39b_4d37_8df8_95ddd910cae2.slice/crio-64bd9028b1fc0a40f080e13e41173ed80398bb65173ba2ceca905eb8c885011e WatchSource:0}: Error finding container 64bd9028b1fc0a40f080e13e41173ed80398bb65173ba2ceca905eb8c885011e: Status 404 returned error can't find the container with id 64bd9028b1fc0a40f080e13e41173ed80398bb65173ba2ceca905eb8c885011e
Jul 21 00:42:49.817459 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.817394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" event={"ID":"2f30bb7c-a39b-4d37-8df8-95ddd910cae2","Type":"ContainerStarted","Data":"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"}
Jul 21 00:42:49.817459 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.817457    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" event={"ID":"2f30bb7c-a39b-4d37-8df8-95ddd910cae2","Type":"ContainerStarted","Data":"64bd9028b1fc0a40f080e13e41173ed80398bb65173ba2ceca905eb8c885011e"}
Jul 21 00:42:49.858987 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:49.858930    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" podStartSLOduration=0.858911822 podStartE2EDuration="858.911822ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:42:49.858262702 +0000 UTC m=+1568.442125598" watchObservedRunningTime="2026-07-21 00:42:49.858911822 +0000 UTC m=+1568.442774746"
Jul 21 00:42:59.462086 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:59.462047    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:59.462086 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:59.462095    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:59.474868 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:59.474843    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:42:59.876384 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:42:59.876360    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:43:25.422519 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:25.422484    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:43:40.444129 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.444026    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" podUID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" containerName="main" containerID="cri-o://e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c" gracePeriod=45
Jul 21 00:43:40.695792 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.695731    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:43:40.804258 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804229    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804302    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804334    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804353    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cnv5\" (UniqueName: \"kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804420 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804393    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804602 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804475    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache\") pod \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\" (UID: \"2f30bb7c-a39b-4d37-8df8-95ddd910cae2\") "
Jul 21 00:43:40.804602 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804574    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home" (OuterVolumeSpecName: "home") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:43:40.804702 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804606    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:43:40.804790 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804772    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:40.804847 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804792    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:40.804847 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.804805    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:43:40.806821 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.806789    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:43:40.806821 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.806802    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm" (OuterVolumeSpecName: "dshm") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:43:40.806975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.806842    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5" (OuterVolumeSpecName: "kube-api-access-4cnv5") pod "2f30bb7c-a39b-4d37-8df8-95ddd910cae2" (UID: "2f30bb7c-a39b-4d37-8df8-95ddd910cae2"). InnerVolumeSpecName "kube-api-access-4cnv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:43:40.905790 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.905753    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:40.905965 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.905785    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cnv5\" (UniqueName: \"kubernetes.io/projected/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-kube-api-access-4cnv5\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:40.905965 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.905817    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:40.905965 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:40.905830    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f30bb7c-a39b-4d37-8df8-95ddd910cae2-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:43:41.064117 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.064085    2561 generic.go:356] "Generic (PLEG): container finished" podID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" containerID="e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c" exitCode=0
Jul 21 00:43:41.064290 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.064147    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"
Jul 21 00:43:41.064290 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.064166    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" event={"ID":"2f30bb7c-a39b-4d37-8df8-95ddd910cae2","Type":"ContainerDied","Data":"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"}
Jul 21 00:43:41.064290 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.064198    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn" event={"ID":"2f30bb7c-a39b-4d37-8df8-95ddd910cae2","Type":"ContainerDied","Data":"64bd9028b1fc0a40f080e13e41173ed80398bb65173ba2ceca905eb8c885011e"}
Jul 21 00:43:41.064290 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.064214    2561 scope.go:117] "RemoveContainer" containerID="e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"
Jul 21 00:43:41.073908 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.073888    2561 scope.go:117] "RemoveContainer" containerID="e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"
Jul 21 00:43:41.074176 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:43:41.074157    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c\": container with ID starting with e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c not found: ID does not exist" containerID="e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"
Jul 21 00:43:41.074221 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.074186    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c"} err="failed to get container status \"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c\": rpc error: code = NotFound desc = could not find container \"e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c\": container with ID starting with e8a35449669bae4fdd2715bd68450378566311e814fefe1e6b35eb240d1a7e9c not found: ID does not exist"
Jul 21 00:43:41.085322 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.085297    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:43:41.088014 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.087992    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e58bdff0-kserve-74899699f5-ml5dn"]
Jul 21 00:43:41.947054 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:43:41.947021    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" path="/var/lib/kubelet/pods/2f30bb7c-a39b-4d37-8df8-95ddd910cae2/volumes"
Jul 21 00:46:42.011808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:46:42.011779    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:46:42.020216 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:46:42.020190    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:51:42.046741 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:51:42.046715    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:51:42.054033 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:51:42.054009    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:52:48.644784 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.644748    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:52:48.645323 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.645306    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" containerName="main"
Jul 21 00:52:48.645378 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.645326    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" containerName="main"
Jul 21 00:52:48.645416 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.645410    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f30bb7c-a39b-4d37-8df8-95ddd910cae2" containerName="main"
Jul 21 00:52:48.648621 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.648601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.651016 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.650989    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c6467f0b\"/\"kube-root-ca.crt\""
Jul 21 00:52:48.651016 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.650992    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c6467f0b\"/\"default-dockercfg-qdwmc\""
Jul 21 00:52:48.651926 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.651906    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c6467f0b\"/\"openshift-service-ca.crt\""
Jul 21 00:52:48.652034 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.651942    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c6467f0b\"/\"e2e-lifecycle-model-c6467f0b-kserve-self-signed-certs\""
Jul 21 00:52:48.655246 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.655222    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:52:48.671838 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.671812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.671929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.671853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.671929 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.671905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.672004 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.671931    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.672042 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.672006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmd9w\" (UniqueName: \"kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.672042 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.672025    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772572 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772545    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmd9w\" (UniqueName: \"kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772732 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772577    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772732 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772732 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772732 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.772732 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.772681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.773037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.773013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.773110 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.773078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.773165 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.773113    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.775033 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.775008    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.775396 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.775377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.779682 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.779663    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmd9w\" (UniqueName: \"kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w\") pod \"e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") " pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:48.960416 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:48.960345    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:49.090646 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:49.090513    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:52:49.092800 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:52:49.092764    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0e84014_19fd_4188_bd76_6f049d302eed.slice/crio-1787a44883f89bfed16f82addbe0447845062c882c6ef4b5d60ed3870feadb02 WatchSource:0}: Error finding container 1787a44883f89bfed16f82addbe0447845062c882c6ef4b5d60ed3870feadb02: Status 404 returned error can't find the container with id 1787a44883f89bfed16f82addbe0447845062c882c6ef4b5d60ed3870feadb02
Jul 21 00:52:49.707600 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:49.707564    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" event={"ID":"b0e84014-19fd-4188-bd76-6f049d302eed","Type":"ContainerStarted","Data":"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"}
Jul 21 00:52:49.707980 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:49.707608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" event={"ID":"b0e84014-19fd-4188-bd76-6f049d302eed","Type":"ContainerStarted","Data":"1787a44883f89bfed16f82addbe0447845062c882c6ef4b5d60ed3870feadb02"}
Jul 21 00:52:49.724035 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:49.723980    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" podStartSLOduration=1.723965188 podStartE2EDuration="1.723965188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:52:49.722654287 +0000 UTC m=+2168.306517204" watchObservedRunningTime="2026-07-21 00:52:49.723965188 +0000 UTC m=+2168.307828085"
Jul 21 00:52:58.960988 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:58.960956    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:58.961374 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:58.961002    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:58.973671 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:58.973648    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:52:59.768256 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:52:59.768221    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:53:36.608089 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:36.608056    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:53:51.628490 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:51.628429    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" podUID="b0e84014-19fd-4188-bd76-6f049d302eed" containerName="main" containerID="cri-o://0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39" gracePeriod=45
Jul 21 00:53:51.883567 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:51.883512    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:53:52.010827 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.010798    2561 generic.go:356] "Generic (PLEG): container finished" podID="b0e84014-19fd-4188-bd76-6f049d302eed" containerID="0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39" exitCode=0
Jul 21 00:53:52.010970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.010837    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"
Jul 21 00:53:52.010970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.010862    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" event={"ID":"b0e84014-19fd-4188-bd76-6f049d302eed","Type":"ContainerDied","Data":"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"}
Jul 21 00:53:52.010970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.010889    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx" event={"ID":"b0e84014-19fd-4188-bd76-6f049d302eed","Type":"ContainerDied","Data":"1787a44883f89bfed16f82addbe0447845062c882c6ef4b5d60ed3870feadb02"}
Jul 21 00:53:52.010970 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.010906    2561 scope.go:117] "RemoveContainer" containerID="0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"
Jul 21 00:53:52.019111 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.019094    2561 scope.go:117] "RemoveContainer" containerID="0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"
Jul 21 00:53:52.019370 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:53:52.019352    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39\": container with ID starting with 0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39 not found: ID does not exist" containerID="0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"
Jul 21 00:53:52.019431 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.019386    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39"} err="failed to get container status \"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39\": rpc error: code = NotFound desc = could not find container \"0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39\": container with ID starting with 0b2b83f47394758729ef006cbc919403554ecaaf8f7cb16bba6de7f6151bfd39 not found: ID does not exist"
Jul 21 00:53:52.032643 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032625    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032720 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032672    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmd9w\" (UniqueName: \"kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032759 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032727    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032759 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032750    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032823 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032800    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032867 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032852    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs\") pod \"b0e84014-19fd-4188-bd76-6f049d302eed\" (UID: \"b0e84014-19fd-4188-bd76-6f049d302eed\") "
Jul 21 00:53:52.032908 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032866    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home" (OuterVolumeSpecName: "home") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:53:52.033021 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.032998    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:53:52.033143 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.033077    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:53:52.033207 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.033176    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.033207 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.033192    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.033207 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.033201    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.035027 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.034997    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:53:52.035027 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.035009    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm" (OuterVolumeSpecName: "dshm") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:53:52.035137 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.035096    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w" (OuterVolumeSpecName: "kube-api-access-pmd9w") pod "b0e84014-19fd-4188-bd76-6f049d302eed" (UID: "b0e84014-19fd-4188-bd76-6f049d302eed"). InnerVolumeSpecName "kube-api-access-pmd9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:53:52.133975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.133925    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e84014-19fd-4188-bd76-6f049d302eed-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.133975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.133945    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e84014-19fd-4188-bd76-6f049d302eed-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.133975 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.133955    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmd9w\" (UniqueName: \"kubernetes.io/projected/b0e84014-19fd-4188-bd76-6f049d302eed-kube-api-access-pmd9w\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:53:52.338351 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.338324    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:53:52.346717 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:52.346697    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c6467f0b/e2e-lifecycle-model-c6467f0b-kserve-675cdb44cf-bk6lx"]
Jul 21 00:53:53.946963 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:53:53.946928    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0e84014-19fd-4188-bd76-6f049d302eed" path="/var/lib/kubelet/pods/b0e84014-19fd-4188-bd76-6f049d302eed/volumes"
Jul 21 00:55:37.464754 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.464668    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:55:37.467074 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.465073    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0e84014-19fd-4188-bd76-6f049d302eed" containerName="main"
Jul 21 00:55:37.467074 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.465085    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0e84014-19fd-4188-bd76-6f049d302eed" containerName="main"
Jul 21 00:55:37.467074 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.465164    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0e84014-19fd-4188-bd76-6f049d302eed" containerName="main"
Jul 21 00:55:37.468008 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.467993    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.470470 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.470448    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1c3d09\"/\"openshift-service-ca.crt\""
Jul 21 00:55:37.471580 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.471558    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1c3d09\"/\"kube-root-ca.crt\""
Jul 21 00:55:37.471665 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.471634    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1c3d09\"/\"test-model-1c3d09-kserve-self-signed-certs\""
Jul 21 00:55:37.471714 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.471681    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1c3d09\"/\"default-dockercfg-6r94l\""
Jul 21 00:55:37.477581 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.477561    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:55:37.494091 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.494196 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494098    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mw4d\" (UniqueName: \"kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.494196 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.494196 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.494304 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.494341 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.494308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.595639 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.595808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595645    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mw4d\" (UniqueName: \"kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.595808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.595808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.595808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.596058 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.595823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.596135 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.596111    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.596198 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.596173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.596240 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.596215    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.598037 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.598021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.598303 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.598286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.604116 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.604092    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mw4d\" (UniqueName: \"kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d\") pod \"test-model-1c3d09-kserve-59c47b4d7c-lf726\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") " pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.777943 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.777912    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:37.912328 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:37.912306    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:55:37.914836 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:55:37.914811    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf60c9577_22b0_4910_8d72_dc7e619e59da.slice/crio-6511d01673f52f65d282931aef3a10908fe262584d0289707ac7eb552ac35c24 WatchSource:0}: Error finding container 6511d01673f52f65d282931aef3a10908fe262584d0289707ac7eb552ac35c24: Status 404 returned error can't find the container with id 6511d01673f52f65d282931aef3a10908fe262584d0289707ac7eb552ac35c24
Jul 21 00:55:38.518010 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:38.517973    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" event={"ID":"f60c9577-22b0-4910-8d72-dc7e619e59da","Type":"ContainerStarted","Data":"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"}
Jul 21 00:55:38.518414 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:38.518024    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" event={"ID":"f60c9577-22b0-4910-8d72-dc7e619e59da","Type":"ContainerStarted","Data":"6511d01673f52f65d282931aef3a10908fe262584d0289707ac7eb552ac35c24"}
Jul 21 00:55:38.535013 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:38.534952    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" podStartSLOduration=1.5349380620000002 podStartE2EDuration="1.534938062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:55:38.534397595 +0000 UTC m=+2337.118260492" watchObservedRunningTime="2026-07-21 00:55:38.534938062 +0000 UTC m=+2337.118801246"
Jul 21 00:55:47.778427 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:47.778395    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:47.778852 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:47.778455    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:47.790925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:47.790898    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:55:48.575756 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:55:48.575722    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:56:36.925667 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:36.925625    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:56:42.077107 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:42.077075    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:56:42.084543 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:42.084516    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 00:56:51.950828 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:51.950780    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" podUID="f60c9577-22b0-4910-8d72-dc7e619e59da" containerName="main" containerID="cri-o://f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245" gracePeriod=45
Jul 21 00:56:52.205375 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.205315    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:56:52.376701 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.376667    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.376701 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.376701    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.376919 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.376899    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.376960 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.376943    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mw4d\" (UniqueName: \"kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.377046 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377010    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.377119 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377053    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home\") pod \"f60c9577-22b0-4910-8d72-dc7e619e59da\" (UID: \"f60c9577-22b0-4910-8d72-dc7e619e59da\") "
Jul 21 00:56:52.377202 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377174    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache" (OuterVolumeSpecName: "model-cache") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:56:52.377347 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377296    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:56:52.377584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377381    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home" (OuterVolumeSpecName: "home") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:56:52.377584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377412    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.377584 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.377428    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.379078 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.379058    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 00:56:52.379609 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.379586    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm" (OuterVolumeSpecName: "dshm") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 00:56:52.379697 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.379612    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d" (OuterVolumeSpecName: "kube-api-access-4mw4d") pod "f60c9577-22b0-4910-8d72-dc7e619e59da" (UID: "f60c9577-22b0-4910-8d72-dc7e619e59da"). InnerVolumeSpecName "kube-api-access-4mw4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 00:56:52.478927 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.478826    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mw4d\" (UniqueName: \"kubernetes.io/projected/f60c9577-22b0-4910-8d72-dc7e619e59da-kube-api-access-4mw4d\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.478927 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.478870    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.478927 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.478881    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f60c9577-22b0-4910-8d72-dc7e619e59da-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.478927 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.478890    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f60c9577-22b0-4910-8d72-dc7e619e59da-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 00:56:52.866663 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.866631    2561 generic.go:356] "Generic (PLEG): container finished" podID="f60c9577-22b0-4910-8d72-dc7e619e59da" containerID="f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245" exitCode=0
Jul 21 00:56:52.866818 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.866679    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"
Jul 21 00:56:52.866818 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.866721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" event={"ID":"f60c9577-22b0-4910-8d72-dc7e619e59da","Type":"ContainerDied","Data":"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"}
Jul 21 00:56:52.866818 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.866768    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726" event={"ID":"f60c9577-22b0-4910-8d72-dc7e619e59da","Type":"ContainerDied","Data":"6511d01673f52f65d282931aef3a10908fe262584d0289707ac7eb552ac35c24"}
Jul 21 00:56:52.866818 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.866790    2561 scope.go:117] "RemoveContainer" containerID="f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"
Jul 21 00:56:52.876182 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.876164    2561 scope.go:117] "RemoveContainer" containerID="f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"
Jul 21 00:56:52.876479 ip-10-0-140-105 kubenswrapper[2561]: E0721 00:56:52.876454    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245\": container with ID starting with f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245 not found: ID does not exist" containerID="f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"
Jul 21 00:56:52.876521 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.876492    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245"} err="failed to get container status \"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245\": rpc error: code = NotFound desc = could not find container \"f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245\": container with ID starting with f6a70903870823c41cf0a5810443e9b42bb62c66ca7c906662cbc0440d783245 not found: ID does not exist"
Jul 21 00:56:52.888188 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.888163    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:56:52.892952 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:52.892933    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1c3d09/test-model-1c3d09-kserve-59c47b4d7c-lf726"]
Jul 21 00:56:53.949043 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:56:53.949012    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f60c9577-22b0-4910-8d72-dc7e619e59da" path="/var/lib/kubelet/pods/f60c9577-22b0-4910-8d72-dc7e619e59da/volumes"
Jul 21 00:59:09.866532 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.866494    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 00:59:09.866983 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.866890    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f60c9577-22b0-4910-8d72-dc7e619e59da" containerName="main"
Jul 21 00:59:09.866983 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.866902    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f60c9577-22b0-4910-8d72-dc7e619e59da" containerName="main"
Jul 21 00:59:09.866983 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.866974    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f60c9577-22b0-4910-8d72-dc7e619e59da" containerName="main"
Jul 21 00:59:09.870124 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.870103    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.872717 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.872695    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"auth-test-model-16b734-kserve-self-signed-certs\""
Jul 21 00:59:09.873668 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.873647    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"openshift-service-ca.crt\""
Jul 21 00:59:09.873736 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.873655    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"kube-root-ca.crt\""
Jul 21 00:59:09.873736 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.873719    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"default-dockercfg-pdtsd\""
Jul 21 00:59:09.879152 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.879128    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 00:59:09.953154 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.953331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.953331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.953331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2n6\" (UniqueName: \"kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.953331 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953292    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:09.953494 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:09.953363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.053925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.053886    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2n6\" (UniqueName: \"kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.053925 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.053927    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054184 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.053997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054184 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054184 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054075    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054184 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054505 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054482    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054579 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.054663 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.054645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.056665 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.056645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.056771 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.056753    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.061808 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.061785    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2n6\" (UniqueName: \"kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6\") pod \"auth-test-model-16b734-kserve-849444d4f9-45hqm\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") " pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.180801 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.180710    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:10.325815 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.325790    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 00:59:10.328706 ip-10-0-140-105 kubenswrapper[2561]: W0721 00:59:10.328676    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cb4be05_2568_441e_9515_3b59354f9a86.slice/crio-35e3341470a68b6a9940ffd72c0ea11b2ac21026aa78f8e6942af3b47ba40a44 WatchSource:0}: Error finding container 35e3341470a68b6a9940ffd72c0ea11b2ac21026aa78f8e6942af3b47ba40a44: Status 404 returned error can't find the container with id 35e3341470a68b6a9940ffd72c0ea11b2ac21026aa78f8e6942af3b47ba40a44
Jul 21 00:59:10.525606 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.525561    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" event={"ID":"8cb4be05-2568-441e-9515-3b59354f9a86","Type":"ContainerStarted","Data":"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"}
Jul 21 00:59:10.525786 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.525618    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" event={"ID":"8cb4be05-2568-441e-9515-3b59354f9a86","Type":"ContainerStarted","Data":"35e3341470a68b6a9940ffd72c0ea11b2ac21026aa78f8e6942af3b47ba40a44"}
Jul 21 00:59:10.543368 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:10.543300    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" podStartSLOduration=1.543281029 podStartE2EDuration="1.543281029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 00:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 00:59:10.542265641 +0000 UTC m=+2549.126128538" watchObservedRunningTime="2026-07-21 00:59:10.543281029 +0000 UTC m=+2549.127143927"
Jul 21 00:59:20.181216 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:20.181176    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:20.181216 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:20.181227    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:20.193824 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:20.193795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 00:59:20.587294 ip-10-0-140-105 kubenswrapper[2561]: I0721 00:59:20.587262    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 01:01:42.106330 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:42.106244    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 01:01:42.114958 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:42.114932    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 01:01:43.922043 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:43.922005    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 01:01:54.546736 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.546696    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:01:54.550415 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.550395    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.552910 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.552890    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"sub-test-model-16b734-kserve-self-signed-certs\""
Jul 21 01:01:54.559225 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.559202    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:01:54.591009 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.590970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.591009 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.591017    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.591246 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.591090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.591246 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.591125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s842\" (UniqueName: \"kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.591246 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.591161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.591246 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.591217    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692274 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692241    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692274 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692274    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s842\" (UniqueName: \"kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692558 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692558 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692558 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692558 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692810 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692790    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692874 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692822    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.692915 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.692874    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.694651 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.694630    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.694932 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.694916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.705735 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.705710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s842\" (UniqueName: \"kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842\") pod \"sub-test-model-16b734-kserve-5c5978cc65-672kh\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") " pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.859989 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.859899    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:01:54.998804 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:54.998777    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:01:55.000492 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:01:55.000469    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d51047b_27c5_4541_926b_41b0ffe695dd.slice/crio-8c7698f67062979a4d367d7c4dd7094695673a15b97b2e53b2b54f55732c79d7 WatchSource:0}: Error finding container 8c7698f67062979a4d367d7c4dd7094695673a15b97b2e53b2b54f55732c79d7: Status 404 returned error can't find the container with id 8c7698f67062979a4d367d7c4dd7094695673a15b97b2e53b2b54f55732c79d7
Jul 21 01:01:55.370829 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:55.370789    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" event={"ID":"8d51047b-27c5-4541-926b-41b0ffe695dd","Type":"ContainerStarted","Data":"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"}
Jul 21 01:01:55.371002 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:55.370840    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" event={"ID":"8d51047b-27c5-4541-926b-41b0ffe695dd","Type":"ContainerStarted","Data":"8c7698f67062979a4d367d7c4dd7094695673a15b97b2e53b2b54f55732c79d7"}
Jul 21 01:01:55.386571 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:55.386513    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" podStartSLOduration=1.386493733 podStartE2EDuration="1.386493733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:01:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:01:55.386060134 +0000 UTC m=+2713.969923032" watchObservedRunningTime="2026-07-21 01:01:55.386493733 +0000 UTC m=+2713.970356631"
Jul 21 01:01:58.944374 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:58.944324    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" podUID="8cb4be05-2568-441e-9515-3b59354f9a86" containerName="main" containerID="cri-o://560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135" gracePeriod=45
Jul 21 01:01:59.205257 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.205200    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 01:01:59.234755 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234723    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.234906 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234772    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.234906 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234880    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk2n6\" (UniqueName: \"kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.235031 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234910    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.235031 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234945    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.235031 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.234991    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache\") pod \"8cb4be05-2568-441e-9515-3b59354f9a86\" (UID: \"8cb4be05-2568-441e-9515-3b59354f9a86\") "
Jul 21 01:01:59.235031 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.235008    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home" (OuterVolumeSpecName: "home") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:01:59.235284 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.235260    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:01:59.235341 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.235283    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.235341 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.235318    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache" (OuterVolumeSpecName: "model-cache") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:01:59.237761 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.237729    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm" (OuterVolumeSpecName: "dshm") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:01:59.238009 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.237987    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 01:01:59.238093 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.238051    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6" (OuterVolumeSpecName: "kube-api-access-bk2n6") pod "8cb4be05-2568-441e-9515-3b59354f9a86" (UID: "8cb4be05-2568-441e-9515-3b59354f9a86"). InnerVolumeSpecName "kube-api-access-bk2n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:01:59.336044 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.336014    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.336044 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.336040    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.336044 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.336049    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb4be05-2568-441e-9515-3b59354f9a86-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.336276 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.336057    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk2n6\" (UniqueName: \"kubernetes.io/projected/8cb4be05-2568-441e-9515-3b59354f9a86-kube-api-access-bk2n6\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.336276 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.336067    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb4be05-2568-441e-9515-3b59354f9a86-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:01:59.391448 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.391403    2561 generic.go:356] "Generic (PLEG): container finished" podID="8cb4be05-2568-441e-9515-3b59354f9a86" containerID="560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135" exitCode=0
Jul 21 01:01:59.391605 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.391473    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"
Jul 21 01:01:59.391605 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.391483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" event={"ID":"8cb4be05-2568-441e-9515-3b59354f9a86","Type":"ContainerDied","Data":"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"}
Jul 21 01:01:59.391605 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.391516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm" event={"ID":"8cb4be05-2568-441e-9515-3b59354f9a86","Type":"ContainerDied","Data":"35e3341470a68b6a9940ffd72c0ea11b2ac21026aa78f8e6942af3b47ba40a44"}
Jul 21 01:01:59.391605 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.391533    2561 scope.go:117] "RemoveContainer" containerID="560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"
Jul 21 01:01:59.400684 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.400667    2561 scope.go:117] "RemoveContainer" containerID="560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"
Jul 21 01:01:59.400967 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:01:59.400945    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135\": container with ID starting with 560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135 not found: ID does not exist" containerID="560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"
Jul 21 01:01:59.401072 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.400973    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135"} err="failed to get container status \"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135\": rpc error: code = NotFound desc = could not find container \"560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135\": container with ID starting with 560ae4e6130ab18fbefccf5db8fa1e0402006d98d596e82fd1c3097f928ec135 not found: ID does not exist"
Jul 21 01:01:59.411366 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.411338    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 01:01:59.415282 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.415263    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/auth-test-model-16b734-kserve-849444d4f9-45hqm"]
Jul 21 01:01:59.947110 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:01:59.947074    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb4be05-2568-441e-9515-3b59354f9a86" path="/var/lib/kubelet/pods/8cb4be05-2568-441e-9515-3b59354f9a86/volumes"
Jul 21 01:02:04.860418 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:04.860376    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:02:04.860960 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:04.860482    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:02:04.873173 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:04.873144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:02:05.435573 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:05.435545    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:02:23.155128 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.155094    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:02:23.155635 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.155509    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb4be05-2568-441e-9515-3b59354f9a86" containerName="main"
Jul 21 01:02:23.155635 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.155521    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb4be05-2568-441e-9515-3b59354f9a86" containerName="main"
Jul 21 01:02:23.155635 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.155621    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb4be05-2568-441e-9515-3b59354f9a86" containerName="main"
Jul 21 01:02:23.158504 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.158489    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.160846 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.160825    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"kube-root-ca.crt\""
Jul 21 01:02:23.161987 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.161960    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"sub-test-model-b4ed4b-kserve-self-signed-certs\""
Jul 21 01:02:23.161987 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.161982    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"default-dockercfg-kr9sp\""
Jul 21 01:02:23.162148 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.161971    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"openshift-service-ca.crt\""
Jul 21 01:02:23.167682 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.167658    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:02:23.270306 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.270501 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4jc\" (UniqueName: \"kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.270501 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270395    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.270501 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.270621 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.270621 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.270571    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371085 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371048    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371265 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371095    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371265 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371265 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4jc\" (UniqueName: \"kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371423 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371423 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371703 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371684    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371773 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.371773 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.371753    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.373643 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.373620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.373780 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.373764    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.379275 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.379254    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4jc\" (UniqueName: \"kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc\") pod \"sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") " pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.468529 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.468407    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:23.606132 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:23.606106    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:02:23.608416 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:02:23.608379    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1731f116_fe35_499f_9b81_023ebe08bf3b.slice/crio-5702a8e65283971947c285bf54b7b9adfb4931dca667ac1c065ad243b2b96ada WatchSource:0}: Error finding container 5702a8e65283971947c285bf54b7b9adfb4931dca667ac1c065ad243b2b96ada: Status 404 returned error can't find the container with id 5702a8e65283971947c285bf54b7b9adfb4931dca667ac1c065ad243b2b96ada
Jul 21 01:02:24.520515 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:24.520481    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" event={"ID":"1731f116-fe35-499f-9b81-023ebe08bf3b","Type":"ContainerStarted","Data":"fc4a2ff6c8ab7086138ba3ff91ea28259401137261f022f9acaeb52ac4f5320f"}
Jul 21 01:02:24.520921 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:24.520526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" event={"ID":"1731f116-fe35-499f-9b81-023ebe08bf3b","Type":"ContainerStarted","Data":"5702a8e65283971947c285bf54b7b9adfb4931dca667ac1c065ad243b2b96ada"}
Jul 21 01:02:24.539078 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:24.539029    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" podStartSLOduration=1.5390125700000001 podStartE2EDuration="1.53901257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:02:24.536534083 +0000 UTC m=+2743.120396995" watchObservedRunningTime="2026-07-21 01:02:24.53901257 +0000 UTC m=+2743.122875468"
Jul 21 01:02:33.468885 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:33.468834    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:33.469302 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:33.469050    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:33.481784 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:33.481759    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:02:33.576677 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:02:33.576646    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:03:09.733035 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:09.732958    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:03:10.665239 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:10.665208    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:03:24.755982 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:24.755941    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" podUID="8d51047b-27c5-4541-926b-41b0ffe695dd" containerName="main" containerID="cri-o://5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7" gracePeriod=45
Jul 21 01:03:25.011820 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.011799    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:03:25.035088 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035060    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s842\" (UniqueName: \"kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035231 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035110    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035231 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035141    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035231 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035156    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035231 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035212    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035456 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035303    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home\") pod \"8d51047b-27c5-4541-926b-41b0ffe695dd\" (UID: \"8d51047b-27c5-4541-926b-41b0ffe695dd\") "
Jul 21 01:03:25.035456 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035409    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:25.036037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035569    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:25.036037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035631    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home" (OuterVolumeSpecName: "home") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:25.036037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035970    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.036037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.035992    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.036037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.036004    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.037555 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.037525    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm" (OuterVolumeSpecName: "dshm") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:25.037854 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.037824    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 01:03:25.037947 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.037891    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842" (OuterVolumeSpecName: "kube-api-access-6s842") pod "8d51047b-27c5-4541-926b-41b0ffe695dd" (UID: "8d51047b-27c5-4541-926b-41b0ffe695dd"). InnerVolumeSpecName "kube-api-access-6s842". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:03:25.137129 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.137079    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s842\" (UniqueName: \"kubernetes.io/projected/8d51047b-27c5-4541-926b-41b0ffe695dd-kube-api-access-6s842\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.137129 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.137120    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d51047b-27c5-4541-926b-41b0ffe695dd-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.137129 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.137135    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d51047b-27c5-4541-926b-41b0ffe695dd-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:25.686547 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.686480    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" podUID="1731f116-fe35-499f-9b81-023ebe08bf3b" containerName="main" containerID="cri-o://fc4a2ff6c8ab7086138ba3ff91ea28259401137261f022f9acaeb52ac4f5320f" gracePeriod=45
Jul 21 01:03:25.832268 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.832235    2561 generic.go:356] "Generic (PLEG): container finished" podID="1731f116-fe35-499f-9b81-023ebe08bf3b" containerID="fc4a2ff6c8ab7086138ba3ff91ea28259401137261f022f9acaeb52ac4f5320f" exitCode=0
Jul 21 01:03:25.832710 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.832311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" event={"ID":"1731f116-fe35-499f-9b81-023ebe08bf3b","Type":"ContainerDied","Data":"fc4a2ff6c8ab7086138ba3ff91ea28259401137261f022f9acaeb52ac4f5320f"}
Jul 21 01:03:25.833705 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.833682    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d51047b-27c5-4541-926b-41b0ffe695dd" containerID="5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7" exitCode=0
Jul 21 01:03:25.833824 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.833748    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" event={"ID":"8d51047b-27c5-4541-926b-41b0ffe695dd","Type":"ContainerDied","Data":"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"}
Jul 21 01:03:25.833824 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.833782    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh" event={"ID":"8d51047b-27c5-4541-926b-41b0ffe695dd","Type":"ContainerDied","Data":"8c7698f67062979a4d367d7c4dd7094695673a15b97b2e53b2b54f55732c79d7"}
Jul 21 01:03:25.833824 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.833800    2561 scope.go:117] "RemoveContainer" containerID="5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"
Jul 21 01:03:25.833971 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.833752    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"
Jul 21 01:03:25.848747 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.848723    2561 scope.go:117] "RemoveContainer" containerID="5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"
Jul 21 01:03:25.849102 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:03:25.849066    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7\": container with ID starting with 5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7 not found: ID does not exist" containerID="5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"
Jul 21 01:03:25.849193 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.849106    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7"} err="failed to get container status \"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7\": rpc error: code = NotFound desc = could not find container \"5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7\": container with ID starting with 5245fe4e45378bdacac83001c658b3741839aaf23ec29605fbe1cdcd3e4c2eb7 not found: ID does not exist"
Jul 21 01:03:25.858171 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.858143    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:03:25.862264 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.862239    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/sub-test-model-16b734-kserve-5c5978cc65-672kh"]
Jul 21 01:03:25.947785 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.947753    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d51047b-27c5-4541-926b-41b0ffe695dd" path="/var/lib/kubelet/pods/8d51047b-27c5-4541-926b-41b0ffe695dd/volumes"
Jul 21 01:03:25.955395 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:25.955376    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:03:26.045963 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.045925    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046157 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046008    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046157 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046049    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046157 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046101    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046157 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046140    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk4jc\" (UniqueName: \"kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046362 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm\") pod \"1731f116-fe35-499f-9b81-023ebe08bf3b\" (UID: \"1731f116-fe35-499f-9b81-023ebe08bf3b\") "
Jul 21 01:03:26.046362 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046241    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:26.046362 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046299    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home" (OuterVolumeSpecName: "home") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:26.046533 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:26.046583 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046557    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.046583 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046570    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.046644 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.046582    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.048660 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.048632    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 01:03:26.048775 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.048662    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm" (OuterVolumeSpecName: "dshm") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:03:26.048775 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.048732    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc" (OuterVolumeSpecName: "kube-api-access-xk4jc") pod "1731f116-fe35-499f-9b81-023ebe08bf3b" (UID: "1731f116-fe35-499f-9b81-023ebe08bf3b"). InnerVolumeSpecName "kube-api-access-xk4jc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:03:26.148056 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.148016    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1731f116-fe35-499f-9b81-023ebe08bf3b-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.148056 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.148049    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk4jc\" (UniqueName: \"kubernetes.io/projected/1731f116-fe35-499f-9b81-023ebe08bf3b-kube-api-access-xk4jc\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.148056 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.148060    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1731f116-fe35-499f-9b81-023ebe08bf3b-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:03:26.839598 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.839569    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"
Jul 21 01:03:26.839598 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.839578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt" event={"ID":"1731f116-fe35-499f-9b81-023ebe08bf3b","Type":"ContainerDied","Data":"5702a8e65283971947c285bf54b7b9adfb4931dca667ac1c065ad243b2b96ada"}
Jul 21 01:03:26.840141 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.839635    2561 scope.go:117] "RemoveContainer" containerID="fc4a2ff6c8ab7086138ba3ff91ea28259401137261f022f9acaeb52ac4f5320f"
Jul 21 01:03:26.862835 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.862808    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:03:26.866877 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:26.866853    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/sub-test-model-b4ed4b-kserve-58b56cd99c-64gpt"]
Jul 21 01:03:27.947552 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:27.947521    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1731f116-fe35-499f-9b81-023ebe08bf3b" path="/var/lib/kubelet/pods/1731f116-fe35-499f-9b81-023ebe08bf3b/volumes"
Jul 21 01:03:33.452564 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.452525    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:03:33.453147 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453051    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d51047b-27c5-4541-926b-41b0ffe695dd" containerName="main"
Jul 21 01:03:33.453147 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453073    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d51047b-27c5-4541-926b-41b0ffe695dd" containerName="main"
Jul 21 01:03:33.453147 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453086    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1731f116-fe35-499f-9b81-023ebe08bf3b" containerName="main"
Jul 21 01:03:33.453147 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453096    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1731f116-fe35-499f-9b81-023ebe08bf3b" containerName="main"
Jul 21 01:03:33.453368 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453208    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d51047b-27c5-4541-926b-41b0ffe695dd" containerName="main"
Jul 21 01:03:33.453368 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.453224    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1731f116-fe35-499f-9b81-023ebe08bf3b" containerName="main"
Jul 21 01:03:33.457660 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.457638    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.460011 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.459989    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"openshift-service-ca.crt\""
Jul 21 01:03:33.460422 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.460406    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"kube-root-ca.crt\""
Jul 21 01:03:33.461236 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.461219    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"default-dockercfg-pdtsd\""
Jul 21 01:03:33.461322 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.461220    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-16b734\"/\"rate-test-model-16b734-kserve-self-signed-certs\""
Jul 21 01:03:33.466106 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.466084    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:03:33.512861 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512827    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.513039 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.513039 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512901    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.513039 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512921    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.513039 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512938    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.513039 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.512985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s9zj\" (UniqueName: \"kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614104 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614071    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614300 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614300 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614130    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614300 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614300 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614162    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614300 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614213    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s9zj\" (UniqueName: \"kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614608 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614666 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614617    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.614705 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.614674    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.616589 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.616570    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.616739 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.616723    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.621707 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.621688    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s9zj\" (UniqueName: \"kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj\") pod \"rate-test-model-16b734-kserve-7896dfbdbc-8fxgz\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") " pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.768258 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.768223    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:33.905716 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:33.905688    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:03:33.908063 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:03:33.908039    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7543c189_d4b2_4149_853a_4aeee133890a.slice/crio-b300204cbb1b8f32b8565859dc35389314cd861e2b2723d7af07343aaff30e10 WatchSource:0}: Error finding container b300204cbb1b8f32b8565859dc35389314cd861e2b2723d7af07343aaff30e10: Status 404 returned error can't find the container with id b300204cbb1b8f32b8565859dc35389314cd861e2b2723d7af07343aaff30e10
Jul 21 01:03:34.884145 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:34.884105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" event={"ID":"7543c189-d4b2-4149-853a-4aeee133890a","Type":"ContainerStarted","Data":"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"}
Jul 21 01:03:34.884145 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:34.884153    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" event={"ID":"7543c189-d4b2-4149-853a-4aeee133890a","Type":"ContainerStarted","Data":"b300204cbb1b8f32b8565859dc35389314cd861e2b2723d7af07343aaff30e10"}
Jul 21 01:03:34.902584 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:34.902534    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" podStartSLOduration=1.902517996 podStartE2EDuration="1.902517996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:03:34.902055085 +0000 UTC m=+2813.485917984" watchObservedRunningTime="2026-07-21 01:03:34.902517996 +0000 UTC m=+2813.486380922"
Jul 21 01:03:43.768956 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:43.768915    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:43.769522 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:43.768975    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:43.781568 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:43.781544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:03:43.938384 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:03:43.938355    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:04:04.251230 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.251193    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:04:04.255232 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.255211    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.259034 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.258981    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"kube-root-ca.crt\""
Jul 21 01:04:04.259169 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.259056    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"rate-test-model-b4ed4b-kserve-self-signed-certs\""
Jul 21 01:04:04.259169 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.259098    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"openshift-service-ca.crt\""
Jul 21 01:04:04.259579 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.259561    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b4ed4b\"/\"default-dockercfg-kr9sp\""
Jul 21 01:04:04.261414 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.261389    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:04:04.296236 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.296370 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296240    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.296370 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296283    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.296370 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.296370 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6bbq\" (UniqueName: \"kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.296531 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.296419    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397259 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397226    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397425 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397287    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397425 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397551 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397551 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397508    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397551 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397531    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6bbq\" (UniqueName: \"kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397713 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397835 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397816    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.397922 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.397821    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.399859 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.399840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.399983 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.399966    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.405595 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.405570    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6bbq\" (UniqueName: \"kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq\") pod \"rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") " pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.565853 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.565820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:04.697958 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:04.697928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:04:04.699955 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:04:04.699925    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f5f574a_12a7_45d2_bf9a_2ba31d374b4e.slice/crio-eff99e46d7f8a0cd6deabe13f1d5154bda2baedcbcc9abc2443862f1801203ea WatchSource:0}: Error finding container eff99e46d7f8a0cd6deabe13f1d5154bda2baedcbcc9abc2443862f1801203ea: Status 404 returned error can't find the container with id eff99e46d7f8a0cd6deabe13f1d5154bda2baedcbcc9abc2443862f1801203ea
Jul 21 01:04:05.024185 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:05.024151    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" event={"ID":"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e","Type":"ContainerStarted","Data":"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"}
Jul 21 01:04:05.024185 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:05.024197    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" event={"ID":"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e","Type":"ContainerStarted","Data":"eff99e46d7f8a0cd6deabe13f1d5154bda2baedcbcc9abc2443862f1801203ea"}
Jul 21 01:04:05.041762 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:05.041709    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" podStartSLOduration=1.041693975 podStartE2EDuration="1.041693975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:04:05.040623921 +0000 UTC m=+2843.624486819" watchObservedRunningTime="2026-07-21 01:04:05.041693975 +0000 UTC m=+2843.625556871"
Jul 21 01:04:14.566611 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:14.566570    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:14.567098 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:14.566627    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:14.579401 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:14.579376    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:15.087035 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:15.087002    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:04:45.745680 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:45.745602    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:04:46.633232 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:46.633203    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:04:47.656326 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.656289    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7191-140002104613424"]
Jul 21 01:04:47.659831 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.659814    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:47.662099 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.662071    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4xsqr\""
Jul 21 01:04:47.666243 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.666182    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104613424"]
Jul 21 01:04:47.783923 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.783892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzdc\" (UniqueName: \"kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc\") pod \"test-curl-7191-140002104613424\" (UID: \"4b75cf69-8b7c-4f93-9bf3-42a37827cab3\") " pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:47.884819 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.884786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzdc\" (UniqueName: \"kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc\") pod \"test-curl-7191-140002104613424\" (UID: \"4b75cf69-8b7c-4f93-9bf3-42a37827cab3\") " pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:47.893070 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.893039    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzdc\" (UniqueName: \"kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc\") pod \"test-curl-7191-140002104613424\" (UID: \"4b75cf69-8b7c-4f93-9bf3-42a37827cab3\") " pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:47.970479 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:47.970420    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:48.106332 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:48.106310    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104613424"]
Jul 21 01:04:48.109646 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:04:48.109621    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b75cf69_8b7c_4f93_9bf3_42a37827cab3.slice/crio-227ca351443366abbc0443e6aa33d4fc9c474a2eca02f3f7823b664565cee798 WatchSource:0}: Error finding container 227ca351443366abbc0443e6aa33d4fc9c474a2eca02f3f7823b664565cee798: Status 404 returned error can't find the container with id 227ca351443366abbc0443e6aa33d4fc9c474a2eca02f3f7823b664565cee798
Jul 21 01:04:48.110905 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:48.110888    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 01:04:48.243253 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:48.243176    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104613424" event={"ID":"4b75cf69-8b7c-4f93-9bf3-42a37827cab3","Type":"ContainerStarted","Data":"227ca351443366abbc0443e6aa33d4fc9c474a2eca02f3f7823b664565cee798"}
Jul 21 01:04:49.251000 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:49.250919    2561 generic.go:356] "Generic (PLEG): container finished" podID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" containerID="0bcd2edf573357e95b4990b53b7cb98d476fa17f289012a1344cb2a597b373ce" exitCode=0
Jul 21 01:04:49.251000 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:49.250980    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104613424" event={"ID":"4b75cf69-8b7c-4f93-9bf3-42a37827cab3","Type":"ContainerDied","Data":"0bcd2edf573357e95b4990b53b7cb98d476fa17f289012a1344cb2a597b373ce"}
Jul 21 01:04:50.390088 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.390066    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:50.426303 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.426278    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7191-140002104613424_4b75cf69-8b7c-4f93-9bf3-42a37827cab3/test-curl-7191-140002104613424/0.log"
Jul 21 01:04:50.451719 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.451684    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7191-140002104613424"]
Jul 21 01:04:50.455652 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.455628    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7191-140002104613424"]
Jul 21 01:04:50.506551 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.506495    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glzdc\" (UniqueName: \"kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc\") pod \"4b75cf69-8b7c-4f93-9bf3-42a37827cab3\" (UID: \"4b75cf69-8b7c-4f93-9bf3-42a37827cab3\") "
Jul 21 01:04:50.508633 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.508611    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc" (OuterVolumeSpecName: "kube-api-access-glzdc") pod "4b75cf69-8b7c-4f93-9bf3-42a37827cab3" (UID: "4b75cf69-8b7c-4f93-9bf3-42a37827cab3"). InnerVolumeSpecName "kube-api-access-glzdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:04:50.588241 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.588216    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7191-140002104613552"]
Jul 21 01:04:50.588599 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.588586    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" containerName="test-curl-7191-140002104613424"
Jul 21 01:04:50.588654 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.588601    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" containerName="test-curl-7191-140002104613424"
Jul 21 01:04:50.588690 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.588663    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" containerName="test-curl-7191-140002104613424"
Jul 21 01:04:50.591323 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.591308    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:50.597498 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.597476    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104613552"]
Jul 21 01:04:50.607980 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.607951    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glzdc\" (UniqueName: \"kubernetes.io/projected/4b75cf69-8b7c-4f93-9bf3-42a37827cab3-kube-api-access-glzdc\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:04:50.708692 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.708668    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xpm\" (UniqueName: \"kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm\") pod \"test-curl-7191-140002104613552\" (UID: \"de01a496-5f88-4d5c-83e6-230439f3220c\") " pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:50.809063 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.809035    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xpm\" (UniqueName: \"kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm\") pod \"test-curl-7191-140002104613552\" (UID: \"de01a496-5f88-4d5c-83e6-230439f3220c\") " pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:50.816926 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.816908    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xpm\" (UniqueName: \"kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm\") pod \"test-curl-7191-140002104613552\" (UID: \"de01a496-5f88-4d5c-83e6-230439f3220c\") " pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:50.901482 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:50.901458    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:51.029849 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:51.029826    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104613552"]
Jul 21 01:04:51.031694 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:04:51.031669    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde01a496_5f88_4d5c_83e6_230439f3220c.slice/crio-1dcd8dbcc19a96d621da9621021c812af3dc0c2768611e2d30fa3ea0a737471c WatchSource:0}: Error finding container 1dcd8dbcc19a96d621da9621021c812af3dc0c2768611e2d30fa3ea0a737471c: Status 404 returned error can't find the container with id 1dcd8dbcc19a96d621da9621021c812af3dc0c2768611e2d30fa3ea0a737471c
Jul 21 01:04:51.262132 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:51.262110    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:51.262391 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:51.262108    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="227ca351443366abbc0443e6aa33d4fc9c474a2eca02f3f7823b664565cee798"
Jul 21 01:04:51.263240 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:51.263212    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104613552" event={"ID":"de01a496-5f88-4d5c-83e6-230439f3220c","Type":"ContainerStarted","Data":"1dcd8dbcc19a96d621da9621021c812af3dc0c2768611e2d30fa3ea0a737471c"}
Jul 21 01:04:51.272497 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:04:51.272473    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7191-140002104613424\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:51.945682 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:04:51.945646    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7191-140002104613424\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" pod="opendatahub/test-curl-7191-140002104613424"
Jul 21 01:04:51.947585 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:51.947562    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b75cf69-8b7c-4f93-9bf3-42a37827cab3" path="/var/lib/kubelet/pods/4b75cf69-8b7c-4f93-9bf3-42a37827cab3/volumes"
Jul 21 01:04:52.269489 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:52.269423    2561 generic.go:356] "Generic (PLEG): container finished" podID="de01a496-5f88-4d5c-83e6-230439f3220c" containerID="eb1ef3a84102071456f25426f157ff98e22647db8334fba14f395161a04a9d8e" exitCode=0
Jul 21 01:04:52.269645 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:52.269510    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104613552" event={"ID":"de01a496-5f88-4d5c-83e6-230439f3220c","Type":"ContainerDied","Data":"eb1ef3a84102071456f25426f157ff98e22647db8334fba14f395161a04a9d8e"}
Jul 21 01:04:53.404545 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.404522    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:53.423916 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.423893    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7191-140002104613552_de01a496-5f88-4d5c-83e6-230439f3220c/test-curl-7191-140002104613552/0.log"
Jul 21 01:04:53.448417 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.448386    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7191-140002104613552"]
Jul 21 01:04:53.451412 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.451391    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7191-140002104613552"]
Jul 21 01:04:53.531666 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.531604    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5xpm\" (UniqueName: \"kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm\") pod \"de01a496-5f88-4d5c-83e6-230439f3220c\" (UID: \"de01a496-5f88-4d5c-83e6-230439f3220c\") "
Jul 21 01:04:53.533818 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.533791    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm" (OuterVolumeSpecName: "kube-api-access-r5xpm") pod "de01a496-5f88-4d5c-83e6-230439f3220c" (UID: "de01a496-5f88-4d5c-83e6-230439f3220c"). InnerVolumeSpecName "kube-api-access-r5xpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:04:53.632360 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.632325    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5xpm\" (UniqueName: \"kubernetes.io/projected/de01a496-5f88-4d5c-83e6-230439f3220c-kube-api-access-r5xpm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:04:53.948745 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:53.948707    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de01a496-5f88-4d5c-83e6-230439f3220c" path="/var/lib/kubelet/pods/de01a496-5f88-4d5c-83e6-230439f3220c/volumes"
Jul 21 01:04:54.281520 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:54.281494    2561 scope.go:117] "RemoveContainer" containerID="eb1ef3a84102071456f25426f157ff98e22647db8334fba14f395161a04a9d8e"
Jul 21 01:04:54.281688 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:54.281498    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104613552"
Jul 21 01:04:59.492490 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.492454    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:04:59.493016 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.492996    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de01a496-5f88-4d5c-83e6-230439f3220c" containerName="test-curl-7191-140002104613552"
Jul 21 01:04:59.493105 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.493019    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de01a496-5f88-4d5c-83e6-230439f3220c" containerName="test-curl-7191-140002104613552"
Jul 21 01:04:59.493162 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.493127    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de01a496-5f88-4d5c-83e6-230439f3220c" containerName="test-curl-7191-140002104613552"
Jul 21 01:04:59.497687 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.497664    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:04:59.500168 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.500141    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4xsqr\""
Jul 21 01:04:59.501702 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.501677    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:04:59.683531 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.683490    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scx77\" (UniqueName: \"kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77\") pod \"test-curl-7191-140002104779808\" (UID: \"f89896af-66a1-4510-b143-28f167f17bfa\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:04:59.784288 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.784254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scx77\" (UniqueName: \"kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77\") pod \"test-curl-7191-140002104779808\" (UID: \"f89896af-66a1-4510-b143-28f167f17bfa\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:04:59.791818 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.791794    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scx77\" (UniqueName: \"kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77\") pod \"test-curl-7191-140002104779808\" (UID: \"f89896af-66a1-4510-b143-28f167f17bfa\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:04:59.807698 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.807676    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:04:59.946299 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:04:59.946269    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf89896af_66a1_4510_b143_28f167f17bfa.slice/crio-29a9b0c41e595fd7dfabbe37cce183b8be0eefd1484ecf2ae688b9266cbd17e0 WatchSource:0}: Error finding container 29a9b0c41e595fd7dfabbe37cce183b8be0eefd1484ecf2ae688b9266cbd17e0: Status 404 returned error can't find the container with id 29a9b0c41e595fd7dfabbe37cce183b8be0eefd1484ecf2ae688b9266cbd17e0
Jul 21 01:04:59.948280 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:04:59.948243    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:00.314499 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:00.314467    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104779808" event={"ID":"f89896af-66a1-4510-b143-28f167f17bfa","Type":"ContainerStarted","Data":"ef41613bf00d1a0239d8713c43418af4a1ebaa0b838ca453a73557225a2b33b9"}
Jul 21 01:05:00.314626 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:00.314518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104779808" event={"ID":"f89896af-66a1-4510-b143-28f167f17bfa","Type":"ContainerStarted","Data":"29a9b0c41e595fd7dfabbe37cce183b8be0eefd1484ecf2ae688b9266cbd17e0"}
Jul 21 01:05:00.328988 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:00.328911    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7191-140002104779808" podStartSLOduration=1.046301169 podStartE2EDuration="1.328895178s" totalImagesPullingTime="282.594009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:05:00.327243856 +0000 UTC m=+2898.911106751" watchObservedRunningTime="2026-07-21 01:05:00.328895178 +0000 UTC m=+2898.912758079"
Jul 21 01:05:00.421496 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:00.421472    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7191-140002104779808_f89896af-66a1-4510-b143-28f167f17bfa/test-curl-7191-140002104779808/0.log"
Jul 21 01:05:00.766177 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:00.766132    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" podUID="7543c189-d4b2-4149-853a-4aeee133890a" containerName="main" containerID="cri-o://adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93" gracePeriod=45
Jul 21 01:05:01.022647 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.022593    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:05:01.095158 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095127    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.095158 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.095386 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095180    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s9zj\" (UniqueName: \"kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.095386 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095208    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.095386 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095338    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.095575 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache" (OuterVolumeSpecName: "model-cache") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:01.095642 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home" (OuterVolumeSpecName: "home") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:01.095732 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095713    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.095806 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.095734    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.097319 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.097293    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj" (OuterVolumeSpecName: "kube-api-access-5s9zj") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "kube-api-access-5s9zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:05:01.097794 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.097771    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm" (OuterVolumeSpecName: "dshm") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:01.097879 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.097796    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 01:05:01.196637 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.196607    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir\") pod \"7543c189-d4b2-4149-853a-4aeee133890a\" (UID: \"7543c189-d4b2-4149-853a-4aeee133890a\") "
Jul 21 01:05:01.196898 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.196877    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7543c189-d4b2-4149-853a-4aeee133890a" (UID: "7543c189-d4b2-4149-853a-4aeee133890a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:01.196953 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.196941    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7543c189-d4b2-4149-853a-4aeee133890a-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.196989 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.196957    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s9zj\" (UniqueName: \"kubernetes.io/projected/7543c189-d4b2-4149-853a-4aeee133890a-kube-api-access-5s9zj\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.196989 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.196967    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.298236 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.298162    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7543c189-d4b2-4149-853a-4aeee133890a-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:01.323496 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.323433    2561 generic.go:356] "Generic (PLEG): container finished" podID="f89896af-66a1-4510-b143-28f167f17bfa" containerID="ef41613bf00d1a0239d8713c43418af4a1ebaa0b838ca453a73557225a2b33b9" exitCode=0
Jul 21 01:05:01.323652 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.323495    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104779808" event={"ID":"f89896af-66a1-4510-b143-28f167f17bfa","Type":"ContainerDied","Data":"ef41613bf00d1a0239d8713c43418af4a1ebaa0b838ca453a73557225a2b33b9"}
Jul 21 01:05:01.324775 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.324753    2561 generic.go:356] "Generic (PLEG): container finished" podID="7543c189-d4b2-4149-853a-4aeee133890a" containerID="adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93" exitCode=0
Jul 21 01:05:01.324872 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.324787    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"
Jul 21 01:05:01.324925 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.324786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" event={"ID":"7543c189-d4b2-4149-853a-4aeee133890a","Type":"ContainerDied","Data":"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"}
Jul 21 01:05:01.324925 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.324910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz" event={"ID":"7543c189-d4b2-4149-853a-4aeee133890a","Type":"ContainerDied","Data":"b300204cbb1b8f32b8565859dc35389314cd861e2b2723d7af07343aaff30e10"}
Jul 21 01:05:01.325052 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.324935    2561 scope.go:117] "RemoveContainer" containerID="adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"
Jul 21 01:05:01.334405 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.334390    2561 scope.go:117] "RemoveContainer" containerID="adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"
Jul 21 01:05:01.334715 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:05:01.334695    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93\": container with ID starting with adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93 not found: ID does not exist" containerID="adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"
Jul 21 01:05:01.334789 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.334722    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93"} err="failed to get container status \"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93\": rpc error: code = NotFound desc = could not find container \"adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93\": container with ID starting with adb31ede94822985baa7b294ecca891246f9e6060061db72c777f7b0d1138d93 not found: ID does not exist"
Jul 21 01:05:01.350068 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.350045    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:05:01.353878 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.353860    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-16b734/rate-test-model-16b734-kserve-7896dfbdbc-8fxgz"]
Jul 21 01:05:01.653721 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.653630    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" podUID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" containerName="main" containerID="cri-o://3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f" gracePeriod=45
Jul 21 01:05:01.904270 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.904217    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:05:01.947598 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:01.947567    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7543c189-d4b2-4149-853a-4aeee133890a" path="/var/lib/kubelet/pods/7543c189-d4b2-4149-853a-4aeee133890a/volumes"
Jul 21 01:05:02.004344 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004300    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004344 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004353    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004645 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004371    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004645 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004462    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6bbq\" (UniqueName: \"kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004645 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004497    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004645 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004573    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home\") pod \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\" (UID: \"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e\") "
Jul 21 01:05:02.004858 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004657    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:02.004858 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004711    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:02.004959 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.004869    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home" (OuterVolumeSpecName: "home") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:02.005079 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.005058    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.005306 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.005084    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.005306 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.005099    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.006742 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.006719    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 01:05:02.006852 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.006804    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq" (OuterVolumeSpecName: "kube-api-access-n6bbq") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "kube-api-access-n6bbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:05:02.006899 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.006866    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm" (OuterVolumeSpecName: "dshm") pod "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" (UID: "8f5f574a-12a7-45d2-bf9a-2ba31d374b4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 01:05:02.105674 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.105625    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6bbq\" (UniqueName: \"kubernetes.io/projected/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-kube-api-access-n6bbq\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.105674 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.105667    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.105674 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.105676    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.332099 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.332063    2561 generic.go:356] "Generic (PLEG): container finished" podID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" containerID="3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f" exitCode=0
Jul 21 01:05:02.332294 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.332113    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"
Jul 21 01:05:02.332294 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.332151    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" event={"ID":"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e","Type":"ContainerDied","Data":"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"}
Jul 21 01:05:02.332294 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.332192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf" event={"ID":"8f5f574a-12a7-45d2-bf9a-2ba31d374b4e","Type":"ContainerDied","Data":"eff99e46d7f8a0cd6deabe13f1d5154bda2baedcbcc9abc2443862f1801203ea"}
Jul 21 01:05:02.332294 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.332210    2561 scope.go:117] "RemoveContainer" containerID="3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"
Jul 21 01:05:02.342626 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.342609    2561 scope.go:117] "RemoveContainer" containerID="3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"
Jul 21 01:05:02.342898 ip-10-0-140-105 kubenswrapper[2561]: E0721 01:05:02.342880    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f\": container with ID starting with 3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f not found: ID does not exist" containerID="3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"
Jul 21 01:05:02.342956 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.342907    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f"} err="failed to get container status \"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f\": rpc error: code = NotFound desc = could not find container \"3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f\": container with ID starting with 3a5e49fbae26b92494a10bc7dbc6eb5b515e52a3c17c12405311715c94582c9f not found: ID does not exist"
Jul 21 01:05:02.354414 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.354390    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:05:02.359019 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.358997    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b4ed4b/rate-test-model-b4ed4b-kserve-55f8bbc547-kgdvf"]
Jul 21 01:05:02.472554 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.472528    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:02.509742 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.509710    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:02.509915 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.509875    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scx77\" (UniqueName: \"kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77\") pod \"f89896af-66a1-4510-b143-28f167f17bfa\" (UID: \"f89896af-66a1-4510-b143-28f167f17bfa\") "
Jul 21 01:05:02.512157 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.512136    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:02.512364 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.512346    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77" (OuterVolumeSpecName: "kube-api-access-scx77") pod "f89896af-66a1-4510-b143-28f167f17bfa" (UID: "f89896af-66a1-4510-b143-28f167f17bfa"). InnerVolumeSpecName "kube-api-access-scx77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:05:02.611483 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.611381    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scx77\" (UniqueName: \"kubernetes.io/projected/f89896af-66a1-4510-b143-28f167f17bfa-kube-api-access-scx77\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:02.642012 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.641976    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:02.642361 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642349    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" containerName="main"
Jul 21 01:05:02.642410 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642363    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" containerName="main"
Jul 21 01:05:02.642410 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642372    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f89896af-66a1-4510-b143-28f167f17bfa" containerName="test-curl-7191-140002104779808"
Jul 21 01:05:02.642410 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642378    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f89896af-66a1-4510-b143-28f167f17bfa" containerName="test-curl-7191-140002104779808"
Jul 21 01:05:02.642410 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642395    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7543c189-d4b2-4149-853a-4aeee133890a" containerName="main"
Jul 21 01:05:02.642410 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642400    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7543c189-d4b2-4149-853a-4aeee133890a" containerName="main"
Jul 21 01:05:02.642587 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642478    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f89896af-66a1-4510-b143-28f167f17bfa" containerName="test-curl-7191-140002104779808"
Jul 21 01:05:02.642587 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642486    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" containerName="main"
Jul 21 01:05:02.642587 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.642494    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7543c189-d4b2-4149-853a-4aeee133890a" containerName="main"
Jul 21 01:05:02.645569 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.645553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:02.652317 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.652293    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:02.711795 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.711761    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgsk9\" (UniqueName: \"kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9\") pod \"test-curl-7191-140002104779808\" (UID: \"8f4da599-64a2-4d90-86c0-76b50eb7b0b5\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:02.813249 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.813203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgsk9\" (UniqueName: \"kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9\") pod \"test-curl-7191-140002104779808\" (UID: \"8f4da599-64a2-4d90-86c0-76b50eb7b0b5\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:02.822474 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.822425    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgsk9\" (UniqueName: \"kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9\") pod \"test-curl-7191-140002104779808\" (UID: \"8f4da599-64a2-4d90-86c0-76b50eb7b0b5\") " pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:02.955594 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:02.955509    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:03.292934 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.292902    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:03.294861 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:05:03.294836    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f4da599_64a2_4d90_86c0_76b50eb7b0b5.slice/crio-4302c02d24452c8051f9cabfd1ad2d0b10745266e541874b40a303c6eb3cd0af WatchSource:0}: Error finding container 4302c02d24452c8051f9cabfd1ad2d0b10745266e541874b40a303c6eb3cd0af: Status 404 returned error can't find the container with id 4302c02d24452c8051f9cabfd1ad2d0b10745266e541874b40a303c6eb3cd0af
Jul 21 01:05:03.339648 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.339627    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29a9b0c41e595fd7dfabbe37cce183b8be0eefd1484ecf2ae688b9266cbd17e0"
Jul 21 01:05:03.339648 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.339645    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:03.340983 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.340953    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104779808" event={"ID":"8f4da599-64a2-4d90-86c0-76b50eb7b0b5","Type":"ContainerStarted","Data":"4302c02d24452c8051f9cabfd1ad2d0b10745266e541874b40a303c6eb3cd0af"}
Jul 21 01:05:03.352240 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.352209    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7191-140002104779808" oldPodUID="f89896af-66a1-4510-b143-28f167f17bfa" podUID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5"
Jul 21 01:05:03.947241 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.947199    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5f574a-12a7-45d2-bf9a-2ba31d374b4e" path="/var/lib/kubelet/pods/8f5f574a-12a7-45d2-bf9a-2ba31d374b4e/volumes"
Jul 21 01:05:03.947632 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:03.947611    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f89896af-66a1-4510-b143-28f167f17bfa" path="/var/lib/kubelet/pods/f89896af-66a1-4510-b143-28f167f17bfa/volumes"
Jul 21 01:05:04.347920 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:04.347882    2561 generic.go:356] "Generic (PLEG): container finished" podID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5" containerID="1e81e72a85041c050295b478ebc4a2a1f40d621fbf5d2efce83561f86c4ee281" exitCode=0
Jul 21 01:05:04.348320 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:04.347972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7191-140002104779808" event={"ID":"8f4da599-64a2-4d90-86c0-76b50eb7b0b5","Type":"ContainerDied","Data":"1e81e72a85041c050295b478ebc4a2a1f40d621fbf5d2efce83561f86c4ee281"}
Jul 21 01:05:05.492851 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.492823    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:05.513642 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.513618    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7191-140002104779808_8f4da599-64a2-4d90-86c0-76b50eb7b0b5/test-curl-7191-140002104779808/0.log"
Jul 21 01:05:05.535366 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.535334    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:05.538603 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.538569    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7191-140002104779808"]
Jul 21 01:05:05.538862 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.538842    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgsk9\" (UniqueName: \"kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9\") pod \"8f4da599-64a2-4d90-86c0-76b50eb7b0b5\" (UID: \"8f4da599-64a2-4d90-86c0-76b50eb7b0b5\") "
Jul 21 01:05:05.541037 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.541005    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9" (OuterVolumeSpecName: "kube-api-access-zgsk9") pod "8f4da599-64a2-4d90-86c0-76b50eb7b0b5" (UID: "8f4da599-64a2-4d90-86c0-76b50eb7b0b5"). InnerVolumeSpecName "kube-api-access-zgsk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 01:05:05.639997 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.639901    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgsk9\" (UniqueName: \"kubernetes.io/projected/8f4da599-64a2-4d90-86c0-76b50eb7b0b5-kube-api-access-zgsk9\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Jul 21 01:05:05.947653 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:05.947572    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5" path="/var/lib/kubelet/pods/8f4da599-64a2-4d90-86c0-76b50eb7b0b5/volumes"
Jul 21 01:05:06.360263 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:06.360232    2561 scope.go:117] "RemoveContainer" containerID="1e81e72a85041c050295b478ebc4a2a1f40d621fbf5d2efce83561f86c4ee281"
Jul 21 01:05:06.360263 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:06.360254    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7191-140002104779808"
Jul 21 01:05:50.000423 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:50.000388    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cd88c9d98-52srp_4b012b1f-5ada-41f4-a0ad-44bc5f3ae073/authorino/0.log"
Jul 21 01:05:56.471874 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:56.471846    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq_d9ed9877-217d-4130-8e3b-4fd091f5e7cf/copy-manifests/0.log"
Jul 21 01:05:56.486006 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:56.485972    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq_d9ed9877-217d-4130-8e3b-4fd091f5e7cf/manager/0.log"
Jul 21 01:05:58.848360 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:58.848328    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cd88c9d98-52srp_4b012b1f-5ada-41f4-a0ad-44bc5f3ae073/authorino/0.log"
Jul 21 01:05:59.201750 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:59.201670    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q54j4_bc91a25a-1bdb-4839-969d-121efe271220/kuadrant-console-plugin/0.log"
Jul 21 01:05:59.452733 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:05:59.452659    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r7b65_e1cd4ba4-a496-4e61-b926-3bf167b70e6d/manager/0.log"
Jul 21 01:06:00.406879 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:00.406848    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66766cb76c-wg8kx_16e4bcfb-1e63-4198-a845-83e6725c1acb/kube-auth-proxy/0.log"
Jul 21 01:06:00.661769 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:00.661690    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jh8tg_20d1ece1-ab82-4ab1-b3ce-804b2612e620/payload-pre-processing/0.log"
Jul 21 01:06:01.295869 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:01.295842    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qc4kh_e96d50bd-0176-4d20-b17a-8218f709db39/main/0.log"
Jul 21 01:06:01.766688 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:01.766663    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-tl978_626db352-8a4a-4041-87f4-b845f33f67b7/main/0.log"
Jul 21 01:06:10.623491 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:10.623387    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t6z2j_3b9d1a2f-bd64-446d-a70d-fbff3a78b59f/global-pull-secret-syncer/0.log"
Jul 21 01:06:10.749432 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:10.749398    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lqp96_f3c649de-ff1c-4205-9d5e-eef443b6cc3b/konnectivity-agent/0.log"
Jul 21 01:06:10.828451 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:10.828422    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-105.ec2.internal_e952599a926de01ba0ecf526bfb18c75/haproxy/0.log"
Jul 21 01:06:15.821240 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:15.821183    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cd88c9d98-52srp_4b012b1f-5ada-41f4-a0ad-44bc5f3ae073/authorino/0.log"
Jul 21 01:06:15.940021 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:15.939994    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q54j4_bc91a25a-1bdb-4839-969d-121efe271220/kuadrant-console-plugin/0.log"
Jul 21 01:06:16.090690 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:16.090601    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r7b65_e1cd4ba4-a496-4e61-b926-3bf167b70e6d/manager/0.log"
Jul 21 01:06:17.809794 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.809766    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/alertmanager/0.log"
Jul 21 01:06:17.846091 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.846067    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/config-reloader/0.log"
Jul 21 01:06:17.875540 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.875512    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/kube-rbac-proxy-web/0.log"
Jul 21 01:06:17.892505 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.892485    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/kube-rbac-proxy/0.log"
Jul 21 01:06:17.924719 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.924685    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/kube-rbac-proxy-metric/0.log"
Jul 21 01:06:17.957114 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.957088    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/prom-label-proxy/0.log"
Jul 21 01:06:17.989011 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:17.988987    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7ac892-bd4b-4868-925d-a1532bfeee53/init-config-reloader/0.log"
Jul 21 01:06:18.027912 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:18.027880    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-cbbjf_ebd52be5-0093-45e9-a9c5-4e89c42d1bd3/cluster-monitoring-operator/0.log"
Jul 21 01:06:18.208843 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:18.208758    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24nj8_456a8cb2-ef02-4ef9-879f-959435f4970c/node-exporter/0.log"
Jul 21 01:06:18.234893 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:18.234868    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24nj8_456a8cb2-ef02-4ef9-879f-959435f4970c/kube-rbac-proxy/0.log"
Jul 21 01:06:18.257983 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:18.257957    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24nj8_456a8cb2-ef02-4ef9-879f-959435f4970c/init-textfile/0.log"
Jul 21 01:06:18.987727 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:18.987699    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-w2rfl_900771d1-6728-485d-b667-906657f967d8/prometheus-operator/0.log"
Jul 21 01:06:19.009182 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:19.009156    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-w2rfl_900771d1-6728-485d-b667-906657f967d8/kube-rbac-proxy/0.log"
Jul 21 01:06:19.069608 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:19.069580    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c56898f4f-rw6r7_fbd21c1e-c5ba-4033-aaeb-a6f9410106c8/telemeter-client/0.log"
Jul 21 01:06:19.094089 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:19.094066    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c56898f4f-rw6r7_fbd21c1e-c5ba-4033-aaeb-a6f9410106c8/reload/0.log"
Jul 21 01:06:19.115704 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:19.115681    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c56898f4f-rw6r7_fbd21c1e-c5ba-4033-aaeb-a6f9410106c8/kube-rbac-proxy/0.log"
Jul 21 01:06:20.445356 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:20.445325    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-8ptrs_46bee106-bf85-4b5f-90a1-ef0cfafdcc86/networking-console-plugin/0.log"
Jul 21 01:06:21.029961 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.029925    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"]
Jul 21 01:06:21.030361 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.030344    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5" containerName="test-curl-7191-140002104779808"
Jul 21 01:06:21.030361 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.030360    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5" containerName="test-curl-7191-140002104779808"
Jul 21 01:06:21.030561 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.030426    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f4da599-64a2-4d90-86c0-76b50eb7b0b5" containerName="test-curl-7191-140002104779808"
Jul 21 01:06:21.033721 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.033700    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.035868 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.035851    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sf6l8\"/\"kube-root-ca.crt\""
Jul 21 01:06:21.036741 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.036721    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sf6l8\"/\"openshift-service-ca.crt\""
Jul 21 01:06:21.036836 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.036746    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sf6l8\"/\"default-dockercfg-pslfp\""
Jul 21 01:06:21.043337 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.043315    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"]
Jul 21 01:06:21.088866 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.088838    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/1.log"
Jul 21 01:06:21.093818 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.093777    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-f8n4k_4e48fe7f-30e5-402f-b4f5-ceaf629ca176/console-operator/2.log"
Jul 21 01:06:21.112152 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.112127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-podres\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.112277 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.112181    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q98d6\" (UniqueName: \"kubernetes.io/projected/0665eeaa-4f6b-450e-ac66-beef10a7e517-kube-api-access-q98d6\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.112345 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.112288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-lib-modules\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.112345 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.112330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-sys\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.112470 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.112415    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-proc\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213381 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213287    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-proc\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213381 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213352    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-proc\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213640 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-podres\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213640 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q98d6\" (UniqueName: \"kubernetes.io/projected/0665eeaa-4f6b-450e-ac66-beef10a7e517-kube-api-access-q98d6\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213640 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213512    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-podres\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213640 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213572    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-lib-modules\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213640 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213600    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-sys\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213888 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213671    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-sys\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.213888 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.213717    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0665eeaa-4f6b-450e-ac66-beef10a7e517-lib-modules\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.221370 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.221347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q98d6\" (UniqueName: \"kubernetes.io/projected/0665eeaa-4f6b-450e-ac66-beef10a7e517-kube-api-access-q98d6\") pod \"perf-node-gather-daemonset-4n8qb\" (UID: \"0665eeaa-4f6b-450e-ac66-beef10a7e517\") " pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.344930 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.344843    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.477635 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.477604    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"]
Jul 21 01:06:21.480890 ip-10-0-140-105 kubenswrapper[2561]: W0721 01:06:21.480862    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0665eeaa_4f6b_450e_ac66_beef10a7e517.slice/crio-e222c24c32dd30df4238c0dce216fb97519c591f4450cba1755f25c4ae509d76 WatchSource:0}: Error finding container e222c24c32dd30df4238c0dce216fb97519c591f4450cba1755f25c4ae509d76: Status 404 returned error can't find the container with id e222c24c32dd30df4238c0dce216fb97519c591f4450cba1755f25c4ae509d76
Jul 21 01:06:21.721946 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.721845    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb" event={"ID":"0665eeaa-4f6b-450e-ac66-beef10a7e517","Type":"ContainerStarted","Data":"8f962c4f496a551cc22c62ecb1b4711d04f452959bbeab60a3c77a215a8b3eec"}
Jul 21 01:06:21.721946 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.721913    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb" event={"ID":"0665eeaa-4f6b-450e-ac66-beef10a7e517","Type":"ContainerStarted","Data":"e222c24c32dd30df4238c0dce216fb97519c591f4450cba1755f25c4ae509d76"}
Jul 21 01:06:21.722151 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.722008    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:21.739052 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:21.739005    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb" podStartSLOduration=0.738990488 podStartE2EDuration="738.990488ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:06:21.73749014 +0000 UTC m=+2980.321353048" watchObservedRunningTime="2026-07-21 01:06:21.738990488 +0000 UTC m=+2980.322853385"
Jul 21 01:06:22.402958 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:22.402932    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-q7hck_1b7c55fb-7095-405c-b7a9-bbbbef6d203c/volume-data-source-validator/0.log"
Jul 21 01:06:23.510192 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:23.510153    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9ctc_c3a03790-c202-4c90-9964-42b29b65a1d6/dns/0.log"
Jul 21 01:06:23.532492 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:23.532465    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9ctc_c3a03790-c202-4c90-9964-42b29b65a1d6/kube-rbac-proxy/0.log"
Jul 21 01:06:23.560166 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:23.560143    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m7f5t_ca7c1b1c-accb-4f4f-9b5e-be8b7c2a8665/dns-node-resolver/0.log"
Jul 21 01:06:24.162143 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:24.162115    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kpx88_8a151b2d-b542-4a6b-99c7-5ce86b3be500/node-ca/0.log"
Jul 21 01:06:25.357042 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:25.357004    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66766cb76c-wg8kx_16e4bcfb-1e63-4198-a845-83e6725c1acb/kube-auth-proxy/0.log"
Jul 21 01:06:25.473950 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:25.473926    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jh8tg_20d1ece1-ab82-4ab1-b3ce-804b2612e620/payload-pre-processing/0.log"
Jul 21 01:06:26.269455 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.269375    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c778b_51ffe5ee-1c35-4eb8-a409-f3281f4b6797/serve-healthcheck-canary/0.log"
Jul 21 01:06:26.713689 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.713607    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-vqlfm_18c75a0d-5efe-450c-baff-342084431ad8/insights-operator/0.log"
Jul 21 01:06:26.714171 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.714148    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-vqlfm_18c75a0d-5efe-450c-baff-342084431ad8/insights-operator/1.log"
Jul 21 01:06:26.737647 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.737615    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6sg77_440398bb-cc81-409c-92a2-127f84da31e7/kube-rbac-proxy/0.log"
Jul 21 01:06:26.759860 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.759830    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6sg77_440398bb-cc81-409c-92a2-127f84da31e7/exporter/0.log"
Jul 21 01:06:26.783203 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:26.783176    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6sg77_440398bb-cc81-409c-92a2-127f84da31e7/extractor/0.log"
Jul 21 01:06:27.738210 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:27.738185    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sf6l8/perf-node-gather-daemonset-4n8qb"
Jul 21 01:06:29.947250 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:29.947225    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq_d9ed9877-217d-4130-8e3b-4fd091f5e7cf/manager/0.log"
Jul 21 01:06:29.968548 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:29.968520    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfdb6d6cb-stbdq_d9ed9877-217d-4130-8e3b-4fd091f5e7cf/copy-manifests/0.log"
Jul 21 01:06:31.363395 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:31.363351    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-75bcdcf5cd-qfvs8_c17a1d09-6454-40c0-a6a8-49133610b017/manager/0.log"
Jul 21 01:06:32.293252 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:32.293220    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66766cb76c-wg8kx_16e4bcfb-1e63-4198-a845-83e6725c1acb/kube-auth-proxy/0.log"
Jul 21 01:06:32.370671 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:32.370642    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jh8tg_20d1ece1-ab82-4ab1-b3ce-804b2612e620/payload-pre-processing/0.log"
Jul 21 01:06:33.725071 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:33.725039    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66766cb76c-wg8kx_16e4bcfb-1e63-4198-a845-83e6725c1acb/kube-auth-proxy/0.log"
Jul 21 01:06:33.809570 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:33.809545    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jh8tg_20d1ece1-ab82-4ab1-b3ce-804b2612e620/payload-pre-processing/0.log"
Jul 21 01:06:36.425247 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:36.425219    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pm9gx_f1ea83d1-93fa-4f51-ac8a-e77d3767d64d/migrator/0.log"
Jul 21 01:06:36.447227 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:36.447198    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pm9gx_f1ea83d1-93fa-4f51-ac8a-e77d3767d64d/graceful-termination/0.log"
Jul 21 01:06:36.802383 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:36.802353    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-sjjsp_0052f0d2-f154-44a7-aea7-40e1086be706/kube-storage-version-migrator-operator/1.log"
Jul 21 01:06:36.803141 ip-10-0-140-105 kubenswrapper[2561]: I0721 01:06:36.803126    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-sjjsp_0052f0d2-f154-44a7-aea7-40e1086be706/kube-storage-version-migrator-operator/0.log"